Fine-tuning large language models on proprietary data is a common way to adapt them to specific domains, but it comes with a privacy risk: the model may memorize private details such as personally identifiable information (PII). A new paper on arXiv tackles this by proposing a pruning technique that targets the specific neurons responsible for leaking such data.

The approach, called joint-embedding-guided pruning, uses embeddings to locate and remove these 'leaky neurons' from the fine-tuned model. The goal is to strip out memorized private information without destroying the useful domain knowledge gained during fine-tuning.

The abstract highlights the tension between domain adaptation and privacy, but the full details of the method and its effectiveness are not yet available from the source. The paper is listed as a new announcement on arXiv, so further results will likely appear in the full version.