By John Werner

Some of the biggest advances today in large language models (LLMs) are driven by challenges, such as those related to privacy. Now, engineers have a way to deal with privacy concerns around sensitive data in AI. It’s called split learning.

I found this explanation of how this works, in terms of forward propagation and backpropagation, from the MIT Media Lab, where some of my colleagues work:

“The outputs at the cut layer are sent to another entity which completes the rest of the training without looking at raw data from any client that holds the raw data. This completes a round of forward propagation without sharing raw data. The gradients are now back propagated again from its last layer until the cut layer in a similar fashion. The gradients at the cut layer (and only these gradients) are sent back to radiology client centers. The rest of back propagation is now completed at the radiology client centers. This process is continued until the distributed split learning network is trained without looking at each other’s raw data.”