Large X models lack a crucial component of “open-source”. Freely redistributable and modifiable for any purpose, sure, but there’s no chance in hell of auditing one, let alone if the training data is kept a secret. It’s literally impossible; human beings cannot look at a trillion weights and biases representing a single highly chaotic, unfathomably complex nonlinear function whose input and output space are the totality of human language/images/etc. and say “yup, looks good to me.” Deep learning models – contrasted with traditional machine learning models – learn their own features which almost 100% of the time would be nonsense to a human. You just have a blob of shareware when you run DeepSeek.
(They also just outright steal from billions of copyright-protected sources to create it, so calling it “open-source” is pretty funny.)
Auditing for bias purposes, yea true. But my primary concern is it having the capability to “phone home” which you don’t really need to audit the model itself to be able to detect or prevent
Large X models lack a crucial component of “open-source”. Freely redistributable and modifiable for any purpose, sure, but there’s no chance in hell of auditing one, let alone if the training data is kept a secret. It’s literally impossible; human beings cannot look at a trillion weights and biases representing a single highly chaotic, unfathomably complex nonlinear function whose input and output space are the totality of human language/images/etc. and say “yup, looks good to me.” Deep learning models – contrasted with traditional machine learning models – learn their own features which almost 100% of the time would be nonsense to a human. You just have a blob of shareware when you run DeepSeek.
(They also just outright steal from billions of copyright-protected sources to create it, so calling it “open-source” is pretty funny.)
Auditing for bias purposes, yea true. But my primary concern is it having the capability to “phone home” which you don’t really need to audit the model itself to be able to detect or prevent
There are a few that are “truly” open like IBM Granite, and a handful of others over the 7B range.