DeepSeek and Security in AI
Recently I was asked to respond to a pretty simple customer question. The question was essentially about the security implication of having DeepSeek on the IBM Watson platform. It was a good question, but the answer is a bit nuanced and requires some explanation. I figured it would be a good topic for a post so I can share this with other people who might also be struggling with the question of adopting new LLM technology, or any AI technology, while trying to keep their data secure.
Why is DeepSeek an Issue?
The DeepSeek model was developed by a Chinese group, which often makes people in Americas and Europe suspicious. In my mind, it’s really not a political or organizational trust issue. From a corporate perspective, I don’t trust ANYONE with my data. Do you want me to send data outside of my firewall? If so, then we need to have a long discussion about encryption, where your model is located, what your policy on data use is, and a host of other things. I am not against using some SaaS service that provides me access to AI functionality, but I am keenly aware that I will not be sharing ANY sensitive data with these types of AI deployments.
So how do I keep my data secure — when I don’t trust anyone? I do it by deploying AI models behind my firewall, and that way my data is never exposed outside of my firewall. All of the usual security that I have (firewalls, network monitoring, etc.) is leveraged to keep my data secure. The same idea holds for me if I am using a trusted cloud provider (where we have things in place to keep their cloud secure and which can effectively be considered to be “behind my firewall”). The key concept is this — You are using the security methods already in place.
So using DeepSeek isn’t any different than using any other LLM out there. I don’t trust ANY of them.
What Makes DeepSeek So Exciting?
You can find a lot of overviews of how it works, and most of them seem to be consistent. Many articles will talk about how “this changes everything”. Over the past couple of weeks you can just watch the AI hype engine spin up. DeepSeek IS impressive — for a few reasons that you don’t read so much about.
It is arguable if the Deep Seek model is “better” than other models. I am not going to go down that rabbit hole in this post. AI experts will have long conversations and write many technically dense papers on the measures and metrics used to test the effectiveness of models. It’s good work, it’s needed work, but in a business setting, it’s not very useful. Benchmarks are (by their nature) artificial and somewhat arbitrary. They assume an idealized use case that is supposed to be representative of the “average” or typical scenario. My business doesn’t run on “typical scenarios”, we have targeted and specific things to accomplish.
The thing that really makes DeepSeek impressive is its size. It’s smaller than your usual LLM. It’s because smaller GPU’s were used to train the DeepSeek model. This has led to a model that is smaller. Smaller means it is easier and cheaper to create the initial LLM. It’s also cheaper to use for inferencing, and because of its smaller size, I can deploy it more easily, using more readily available hardware. Couple this smaller size, with accuracy that is comparable to other LLMs that are larger than DeepSeek, and you get something that appears to be a real step forward.
Answer the Question — What About IBM and DeepSeek?
Now we can circle back to the original question. IBM has been pretty consistent with their response to DeepSeek. IBM has stated that DeepSeek just reinforces our belief that smaller more targeted AI models are the way to move AI ahead in the future. IBM has even announced that they will be hosting distilled models based on DeepSeek on the watsonx.ai platform. It is important to note that these distilled DeepSeek models are non-IBM models and their use is subject solely by the terms of the open source license under which they are released (e.g., no indemnification or warranty). These distilled models do not interact with the DeepSeek mobile application or API (so that means that you can securely deploy them).
Distilled models are smaller versions of the original model, and they effectively allow you to create a smaller version of a model, that you can use for inferencing wherever you want (without the data going to the original LLM). This smaller version is not as accurate as the larger model, but it is often quite close (in terms of accuracy). So now you get to put your model where you want (in a secured zone, where your data is protected), and know that it isn’t sharing your input or output data with anyone else.
Conclusions
So if you’re worried about the security implications of DeepSeek, just remember that it is like ANY other LLM hosted out on the internet. None of them should be trusted — if you want to keep your data secure, don’t transmit it out on the internet. Also remember that distilled models are smaller, and they can be SECURELY deployed on your own network (either on your own hardware, or you own dedicated Cloud). That keeps your data (and the model) protected with the same mechanisms that you use today.
