Open Source and the Future AI

My longtime friend Ibrahim Haddad, Ph.D. the Executive Director of the PyTorch Foundation and the Executive Director of The Linux Foundation 's LF AI & Data Foundation , recently shared this paper from early in 2023, Standing Together on Shared Challenges - Report on the 2023 Open Source Congress. It's a good reminder of the values of open source software that provided the basis for the building of the Internet, cloud computing, and hopefully, in large part, artificial intelligence in the future.

I've gone back and forth on my views on open source over the last few years. It's not a love/hate relationship, but there have been many missteps by companies who have been at odds between their open source software development and business models.

The upside is there's so much good that came from open source - WordPress , Linux, The Apache Software Foundation for various infrastructure projects that we depend on whether we know about it or not. But despite that, there is a downside: many companies have struggled and moved away from open source licenses like HashiCorp . Matt Weinberger has a good overview of this at The New Stack .

Also, this approach addresses many of the biggest concerns around privacy and AGI that are dominating the news today. Here's my take on how companies should move forward with open source and AI.

Leveraging Open Source for Ethical and Explainable AI (XAI)

The problem with many proprietary AI systems is that they are black boxes. Even those deemed safe solutions seem to spit out some pretty wacky answers or even leak data, as we saw from an article last week about research done by Google's DeepMind. An open source approach provides a better oversight of new technology than depending on companies to secure your AI solutions alone. Here's my take on what you should be considering in the development of

Prioritize Collaborative Innovation for Enhanced Explainability

Businesses should actively engage with open source platforms to drive the development of explainable AI. Embracing open source communities, such as those around Hugging Face Face, can provide valuable insights into AI algorithms, making them more transparent and understandable. Collaborate with diverse contributors to access broader expertise, which is essential for demystifying AI decision-making processes. Utilize the tools and techniques developed in these communities to ensure your AI systems are interpretable, building trust among stakeholders and facilitating their use by non-technical team members.

Adopt Community Governance to Ensure Ethical AI

Incorporate open source AI solutions into your business strategy to uphold ethical standards in AI development. Actively participate in open source projects, like LangChain and AI models under bona fide open source licenses, to bring diverse ethical perspectives into your AI systems.

Implement Continuous Oversight for Ongoing AI Improvement

It would be best to adopt a proactive stance in overseeing your AI development by leveraging the transparency and community-driven nature of open source projects. Regularly engage with open source AI platforms, using them as a benchmark for continuously monitoring and improving your AI systems. This approach facilitates quick identification and resolution of performance, ethical, or security issues. Businesses should reference tools like the Hugging Face LLM leaderboard for their AI initiatives, benefiting from their vibrant communities' rapid innovation and improvement. This ensures your AI solutions remain cutting-edge, reliable, and open for review.

Businesses should strategically embrace open source models to develop AI solutions that are not only advanced and efficient but also transparent, ethical, and continually evolving.