On Transforming Manufacturing with IoT and Real-Time Data. Interview with Dheeraj Remella.
“We’re also seeing that the latency involved in going to the cloud is becoming a nuisance and there is a lot of movement to bring intelligence to the edge. In this case, edge doesn’t mean “on the device” because that becomes too narrow a context. The edge is a compute location close to all the components and systems in a process.”
Q1. How has the integration of IoT sensors and real-time data processing transformed traditional manufacturing processes since 2020?
Dheeraj Remella: Since 2020, we’ve seen a significant shift. Manufacturers are moving from reactive to proactive operations. Some of the examples are:
- Predictive maintenance is becoming mainstream
- AI-driven decision-making using ML models for real-time inference
- Ubiquitous connectivity through 5G, eSIM, and LoRaWAN has given rise to more data being collected, giving manufacturers immediate visibility into the current state of assets and warehouse parts for faster issue resolution.
- By proactively managing assets and operations, organizations are enjoying energy savings of around 30%
- Digital Twins are being used in the “industrial metaverse” to ensure that a change would have the intended effect before it is implemented in the real world – i.e. the physical twin world.
We’re also seeing that the latency involved in going to the cloud is becoming a nuisance and there is a lot of movement to bring intelligence to the edge. In this case, edge doesn’t mean “on the device” because that becomes too narrow a context. The edge is a compute location close to all the components and systems in a process. So, the organizations need to go beyond the traditional computation of just the equipment efficiency and instead focus on the more meaningful process efficiency.
Q2. What are the key challenges in managing and analyzing the massive scale of IoT data in real-time for smart manufacturing applications?
Dheeraj Remella: Typically, organizations buy software that has its own data storage, which creates silos between processes and departments. These silos create the “Curse of Babel” problem, where every department and software has its own nomenclature and representation. The lack of a commonly accepted and understood ontology creates a significant bump to overcome before true innovation can be achieved. Additionally, the sheer volume, velocity, and variety of IoT data present significant hurdles. Traditional systems struggle to keep up.
Q3. How are edge computing and cloud-based analytics being leveraged to address latency issues in real-time data processing for manufacturing environments?
Dheeraj Remella: Data value is primarily determined by how fresh it is. Younger data is suitable for faster contextual decisions, while older data is better used as a part of a collective for extracting learnings from the system behavior. Naturally, the younger data needs to be acted upon at the edge and the older data should be collected in the cloud for economies of scale for the non-time-sensitive machine learning initiatives.
In addition, edge computing gives the ability to incorporate various secondary concerns into the data processing:
- Security
- Sustainability
- Sovereignty
- Sessionization and aggregation
- Data thinning to send less and just the relevant data to the cloud.
- Digital twins being activated beyond being just data stores allows intelligent participation in the physical processes
- Elimination of redundant or insignificant ephemeral data
Q4. Can you discuss the role of machine learning algorithms in extracting actionable insights from real-time IoT sensor data in smart factories?
Dheeraj Remella: In a fast-changing, hyperconnected industrial world, machine learning is critical. Once the most appropriate algorithm has been selected, they need to be fed the more recent data continuously to ensure an adaptive approach of the model evolution. This rapidly evolving model can then be fed into the real-time compute layer to make better decisions, decreasing false positives/negatives. These models can:
- Detect anomalies in production processes that would otherwise go unnoticed.
- Predict maintenance needs before failures occur, minimizing downtime.
- Optimize resource allocation and production schedules for peak efficiency.
The beauty of this continuous feedback loop of observe, orient, decide and act (OODA) is that these algorithms continuously learn and improve, boosting the overall efficiency of smart factories. We bring these machine-learning insights directly to bear on real-time data and decisions.
Q5. How are manufacturers balancing the need for real-time data processing with data security and privacy concerns, especially when dealing with sensitive industrial information?
Dheeraj Remella: This is a top priority. Manufacturers are employing several strategies:
- Implementing strong encryption and authentication measures. Post Quantum Encryption is becoming an increasingly interesting discussion, albeit nascent.
- Utilizing edge computing to keep sensitive data local, reducing exposure.
- Developing comprehensive data governance policies.
- Thinking of security and privacy at the initial design.
- Employee training.
- Role-based access control of the data for both personnel and systems.
Q6. What advancements in data integration technologies are enabling manufacturers to combine real-time IoT data with existing business systems for more comprehensive insights?
Dheeraj Remella: Integration is key. There is an adjacent need as well, and that is interoperability.
We’re seeing advancements like:
- Custom APIs and middleware solutions bridging legacy and modern systems.
- Standardized data formats facilitating a homogeneous target data model.
- Unified Namespace architectured de-siloing data between various IT and OT systems.
- IoT platforms are evolving to support diverse data sources and protocols, both old and modern.
- Event-driven Unified Real-time Data Platforms that make cognitive decisions powered by machine learning models on the streaming IoT data combined with near-past context.
Q7. How is real-time data processing improving predictive maintenance capabilities in smart manufacturing, and what impact is this having on reducing downtime and optimizing asset utilization?
Dheeraj Remella: The combination of real-time data and smart use of machine learning is revolutionizing maintenance. Maturation in predictive maintenance is allowing organizations to move away from scheduled maintenance, where maintenance activity is done even though the health has not deteriorated. This proactive approach is drastically reducing downtime and optimizing asset utilization across industries. This shift into need-based predictive maintenance also improves the sustainability stance and narrative at these organizations.
Q8. Can you explain the concept of “digital twins” in manufacturing and how real-time data processing is enhancing their effectiveness?
Dheeraj Remella: Usually you would find digital twins to be pure state stores that record the current state of the physical twin. This approach is quite useful in simulation and industrial metaverse-type scenarios. But, increasing maturity at manufacturing organizations is demanding digital twins to also account for the physical twins’ behavior. More often than not, there is an interest in augmenting the behavior with predictive ML models. Now, they are active participants in day-to-day operations with bi-directional data and control flow.
Real-time data processing is enhancing their effectiveness by:
- Providing up-to-the-second information on asset performance.
- Enabling simulation of different scenarios for optimization.
- Facilitating predictive maintenance and process improvements.
- Invoking actions through actuators/controllers to complete the real-time sense-control loop.
But, to make these usage patterns genuinely effective and impactful, there has to be a paradigm shift to start thinking in terms of edge computing, low latency roundtrips, data, and decision immediacy.
Q9. What strategies are being employed to ensure the scalability and reliability of real-time data processing systems as the number of IoT sensors in manufacturing environments continues to grow?
Dheeraj Remella: Increasing sensors and systems and the mounting need for intelligent automation push the narrative to the edge. While the device edge is too narrow of a context, there is a near-edge tier of computing that has the full system and process content. While there are technologies that can scale to the entire data velocity and quantity without compromising on reliability or resilience, manufacturers have to look into localized computing at the edge so that they are also addressing the timeliness and latency sensitivity of decision and response automation. Manufacturers are:
- Adopting cloud and edge computing architectures for flexible scaling.
- Implementing robust data management strategies.
- Utilizing distributed processing techniques to handle increasing data volumes.
- Data platforms that minimize latency and keep infrastructure needs manageable for environments that do not have the luxury of unlimited hardware.
Q10. How do you see the role of AI and machine learning evolving in augmenting real-time data processing for smart manufacturing by 2026, and what new capabilities might this enable?
Dheeraj Remella: By 2026, AI and machine learning will be even more deeply integrated:
- Advanced AI models will enable more autonomous decision-making in production processes with better false-positive and false-negative recognition.
- Machine learning algorithms will become more sophisticated in predictive analytics and optimization.
- Machine learning would move to the edge as well to take advantage of streaming retraining and complete the learn-predict-act cycles, all at the edge.
- We may see the emergence of self-optimizing production lines that can adapt in real time to changing conditions.
- We may even see a degree of democratization of data, decisions and automation to use natural language interaction and management of systems.
These advancements will lead to unprecedented efficiency, quality control, and responsiveness. With the increasing accuracy in manufacturing, enterprises can enjoy much better waste, power, and equipment management, thus having a better sustainability orientation.
Q11. In what applications do Volt Active Data’s customers utilize real-time data processing in the manufacturing sector?
Dheeraj Remella: Volt Active Data’s customers commonly use it for:
- Predictive Maintenance
- Real-Time Quality Control
- Adaptive Production Optimization
- Supply Chain Monitoring and Optimization
- Real-Time Asset Tracking
- Combining sensors and video data through computer vision for complete observation
Q12. Is Volt Active Data a possible solution for the challenges posed by IoT and machine learning at scale in manufacturing?
Dheeraj Remella: Volt Active Data is designed to meet 4 key requirements without compromising on availability and resiliency:
- Scale (Number of things)
- Speed (the rate at which the things generate data)
- Low latency (How quickly do you need to act on that data)
- Data-Decision Accuracy (How accurate does the correlation data need to be compared to eventually consistent systems)
Often, the decisions would need to be augmented with machine learning model inference in real-time in an event-driven manner. Volt Active Data was built from the ground up to address these requirements at any scale with predictable latency SLAs. Our customers rely on us to make sure their systems don’t miss SLAs or lose data, and also ensure that they can integrate with the appropriate downstream and upstream systems in the most efficient way.
Q13. In your opinion, what are Volt’s three most successful manufacturing use cases?
Dheeraj Remella: The top three applications for Volt Active Data are:
- Predictive Maintenance
- Real-time quality Control for early detection of defects
- Adaptive Production Optimization
There are other applications that can adopt Volt’s values as well, such as reducing the non-productive time of assets by continuous monitoring of the asset conditions, pre-emptive ordering of required spare parts before they are needed, identifying the closest technician that can run the maintenance of the asset and the closest warehouse with all the parts required for the maintenance.
These use cases require the data layer to address:
- Scale, with the ability to handle billions of events per day.
- High performance, with the ability to process hundreds of thousands of events per second.
- Low latency, where the moment of engagement is in single-digit milliseconds.
- The ability to handle complex data.
- The ability to make complex decisions on streaming data.
- Immediate consistency and accuracy.
- No data loss.
- Geographic distribution.
……………………………………………………..
Dheeraj Remella is the Chief Product Officer at Volt Active Data, responsible for technical OEM partnerships and enabling customers to take their next step in data-driven decision-making. Dheeraj has been instrumental in significant customer acquisitions and brings 30 years of experience in creating Enterprise solutions in a variety of industries. Dheeraj is a strong believer in the cross-pollination of ideas and innovation between industries and technologies. Dheeraj holds a bachelor’s degree in computer engineering from Madras University.
……………………….
Related Posts
On Use of Analytics in Products Operations at SAAS companies. Q&A with Haricharan Ramachandra
………………………..
“It’s great to get data, it can be painful to enter data. EHRs benefit all stakeholders tremendously because the information you need is in that one computer in front of you, and a lot of it is structured. The biggest challenge is getting that information in there.”
Q1. How is your long term role Emergency Physician at Brigham and Women’s Hospital helping you with your job as Chief Medical Officer and Director of Clinical Innovation at InterSystems?
Jonathan Teich: It is tremendously helpful. I get to be in the role of the clinician, and I experience so many of the trends, the stresses, the problems and gaps that our customers experience – not only clinical, but also administrative and operational issues. Having this continuous exposure to the ups and downs of the healthcare day illuminates areas where our EHR could help to improve workflow and quality. On every single shift, I see something new that could have an IT solution.
Healthcare worldwide is going through a very stressful time: burnout is increasing among clinicians, patients are having a harder time getting access to care, communication is spotty. At InterSystems we can do something about that! And the AI capabilities that InterSystems IntelliCare brings give us even more capability to address the things that I see in my work as an emergency doc.
Q2. Let’s talk about electronic health records (EHR) and healthcare information systems. What are the main current challenges in using them for a variety of stakeholders, such as clinicians, administrators, and patients?
Jonathan Teich: Simple – it’s great to get data, it can be painful to enter data. EHRs benefit all stakeholders tremendously because the information you need is in that one computer in front of you, and a lot of it is structured. The biggest challenge is getting that information in there. Especially because of regulation and payment requirements in the US and other countries, you have to enter data in different fields, different screens, different formats, often in an unnatural way that doesn’t match the flow of thought that evolves in a typical clinical encounter. There’s way too much clerical busy work that could be automated – and that work leads to stress, take-home work that eats into family time, and burnout.
There are challenges in accessing data, too, and it’s about organization. If you want to know something medically important like “what is this patient’s cardiac risk status?”, you have to go to this page for labs, that page for medications, another page for imaging and procedure results, somewhere else for unstructured insights from previous notes. As a physician, my life is a series of healthcare scenarios and “what do I do next” questions – treat pneumonia, address cardiac risk, find an available bed, balance resources, track an epidemic; for each of these, I want to have all the information I need in front of me, in one view, nicely organized.
Q3. In a recent published paper it is reported that “Many clinicians and analysts find EHRs inflexible”. What is your take on this?
Jonathan Teich: That’s another way of stating some of the challenges I mentioned.
Many EHRs are designed from the database up, not from the workflow down. When I work in the ED, I find that the EHR is very good at handling my one or two most frequent tasks – reading test results and notes, and making note entry somewhat less painful. But it’s hard to bring up information in a different way for a different purpose – show me everything about that patient’s cardiac risk, show me everything I need to make the best treatment plan for the next patient’s stroke, tell me what rehab centers are available for the patient after that. A flexible EHR would be like a good hotel concierge – understand whatever I need to know next, and give me all the right information, in one place and in a usable form.
Q4. What about Data Quality? How do manage possible bias and ensure fairness?
Jonathan Teich: In general, capturing data and displaying it back works pretty well. Data quality is a real problem when trying to do advanced things with the data – doing analytics to understand illness trends, sharing data that came from EHRs with slightly different data models, applying guidelines and rules to improve patient care.
There’s lots of partly-missing data, data items that aren’t filled in, just as a consequence of the fast-paced, often rushed nature of medical care and data entry. Or, sometimes a data element on a form is not interpreted uniformly by the many people who are filling that form. If you build an analytic model, or a clinical decision support rule, that relies on that data, it could give you misleading results and affect your decisions. I remember a time earlier in my career when the lab code for a tuberculosis test was changed, but the rules that rely on it did not change, so incorrect alerts for TB started going out; that could have had serious consequences.
Inaccuracies can lead to bias and inequity — for example, when medical centers serving a high-income group capture more data than centers serving a low-income group, or when we rely on past inequitable practice as the basis for training our future decision rules. That has happened on a number of occasions across our industry. It’s very important to think about these possibilities up front when designing analytics and decision support.
Q5. One key aspect is also Data Privacy. What are the common best practices you are aware of?
Jonathan Teich: Of course, this is a huge issue, and as you know, the solutions are partly about better systems and partly about better human practices. Fundamentally, the goal is that only the right persons see a patient’s information, see only what they need to know, and don’t spread it into other uncontrolled channels.
For the first part, role-based access control is a starting point. Smart access control, giving you easy access to patients with whom you have a defined relationship, is better; it’s a good mix of ease of use and tight privacy control.
For the need-to-know part, there are ways to restrict access to certain elements and sections of the record. For example, administrative personnel commonly cannot see certain clinical data; or some data elements may be legally protected from being used for employment or coverage decisions. On the other hand, it’s problematic to block a clinician’s access to some of a patient’s data, because any data may be important for medical decision making. So, for clinical use, often the best practice is to control at the patient level, not so much the data level.
Q6. Correlative analyses of EHR data with other external data sources can provide significant insights. What are the main challenges here?
Jonathan Teich: It boils down to making sure the data being used is correct and appropriate for the purpose. First, you need high-quality patient matching, to make sure that the EHR and the external source are in fact talking about the same person or the same group. Second, attention to data quality and interoperability is especially important here, since the EHR and the external database were populated for different reasons and may have different understanding of the same data elements.
This sort of combination also calls for extra attention to data privacy in all of the participating systems, so that the combined data doesn’t reveal an individual’s identity or expose private EHR data in unacceptable ways.
You’re right about the important insights that can be gained from such a combination of sources, if you do a good job addressing these challenges. At the population level, multi-source data can be extremely valuable for many uses – such as concentrating opioid-treatment resources in neighborhoods where there is a greater history of overdoses, or developing school lunch support programs in areas where EHR data shows malnutrition-related health problems.
Q7. You have recently launched an AI-powered electronic health record (EHR) and healthcare information system. What is it?
Jonathan Teich: What it is, is amazing. InterSystems IntelliCare is a true game-changer from the point of view of a clinician, a patient, a health quality leader, and more. It is a deep integration of a top-class AI database and engine with a full-function EHR, as well as a digital voice and image capture system – all built for each other. This integration provides true depth in every part of the record, letting you get the benefit of AI for a wide range of applications.
From providing custom summaries for many medical scenarios, to reducing the work of both structured and unstructured documentation and ordering, to streamlining communication, to orchestrating complex workflows, to mining information in text notes, we find that the AI-enhanced integrated record can improve quality and reduce burden in many ways. I said above that EHRs need to gather and present data the way that matches my medical thought process; InterSystems IntelliCare is built to address that need.
Of course, we’re continuing to advance InterSystems IntelliCare beyond the version just announced – the potential range of benefits to the care process is staggering.
Q8. Will the use of AI improve Electronic Health Records? If yes, how?
Jonathan Teich: In a nutshell, I think well-designed generative AI gives us a chance to get back to the kind of practice we once had – more focus on the patient, more time to think and talk, less busy work, better communication, less time staring at the computer screen. It also will allow us to unlock the vital information buried in clinical notes, which is so often overlooked today. It will provide a greater, easier-to-absorb awareness of what’s going on, with a single patient or with an entire population. And one of the greatest benefits will be more usable information going to the patients themselves.
Q9. There seems to be a tension between one hand delivering streamlined workflows, reduced administrative burden, enhanced patient interactions, and improved operational efficiency and on the other hand maintaining rigorous human oversight for accuracy and safety. How do you manage this?
Jonathan Teich: With a healthy dose of skepticism, and a great deal of rigor. We are very concerned about measuring, monitoring, and improving our AI systems, particularly with respect to recall (capturing and providing all of the data that it should, without omissions) and precision (not providing false, incorrect, or “hallucinated” information).
InterSystems has a large quality assurance / testing team that plays a huge role in all of our software development; they have adapted themselves for the particular needs of testing AI, and have developed test suites that we are running constantly as we continue to develop. Guided by this, we have added many guardrails in our software and in our prompt engineering to enhance accuracy and safety.
And yes, it’s always important to make sure that there is a human in the loop at strategic points. We have a variety of forcing functions that ensure that human review and final approval is obtained where needed, especially before data is entered into the record. In general, I much prefer to use AI to reduce work burden by 80% with human-in-the-loop enforced, rather than to reduce burden by 100% but have no oversight; it’s still a tremendous benefit.
Q10. Let’s focus on one of the promises of using AI in this context: Enhanced Patient Engagement. What does it mean in practice?
Jonathan Teich: As I’ve mentioned, direct benefits to patients are among the most promising areas for AI. This is a whole interview in itself, but I can summarize some important elements. As a patient in a clinical or hospital care situation, imagine that you have a constant summary of where you are in your care plan, what’s coming up next, your latest results and vital signs, the results of your CT scans – all organized in a readable and understandable fashion.
We just don’t get enough time to give you that kind of information nowadays, but it’s incredibly important. If you need to ask your practice a medical question, AI can help you compose it and help your medical team respond more quickly; or it could streamline things the next time you need to schedule an appointment. And of course, ambient AI voice dictation means that your doctor will have more time in each visit to interact with you, instead of constantly typing into the record. Besides all that, there are lots of ways that AI can interact with you directly at home, helping you to monitor and optimize your self-care, while always standing ready to send a message to your clinical team if a potential problem is arising.
Qx Anything else you wish to add?
Jonathan Teich: This is a remarkable time in healthcare. I really believe that enhancements in data handling, interoperability, and especially generative AI are going to revolutionize our healthcare lives for the better, benefitting clinicians and health workers and scientists and patients — more than any innovation that has taken place in my career. We must always be careful to protect safety, privacy, and equity; but I believe we can. I believe that the next few years could see enormous positive changes in the work of healthcare, the accessibility of healthcare, and the quality of healthcare.
……………………………………………………..
Dr. Jonathan Teich is Chief Medical Officer and Director of Clinical Innovation for EHRs at InterSystems.
He helps lead vision, design, and thought leadership for solutions to high-priority needs of providers, health systems, and governments worldwide, with particular focus on generative AI, usability and workflow, patient engagement, clinical decision support (CDS), and optimizing how EHRs can help achieve strategic clinical and operational goals.
Dr. Teich is a practicing emergency physician at Brigham and Women’s Hospital and assistant professor at Harvard. He founded the Clinical Informatics R&D department at Partners Healthcare (now Mass General Brigham), developing two generations of innovative electronic health records, computerized provider order entry, knowledge retrieval, and CDS applications, and has authored over one hundred publications and three books in the field. He serves on numerous government and industry panels concerned with CDS, healthcare quality, and workflow design; he has served on the board of directors of AMIA, HIMSS, and the eHealth Initiative. Dr. Teich also volunteers as clinical architect and designer with OpenMRS, an open-source electronic health information system serving over fifty low- and middle-income countries worldwide.
Resources
Electronic Health Records (EHR)-2025 American Medical Association.
Related Interviews
On Data Platforms. Interview with Gokhan Uluderya
On using AI and Data Analytics in Pharmaceutical Research. Interview with Bryn Roberts
………………………..
” The most successful people that I have worked with and the most successful organizations I have been a part of have been the ones that embrace learning and change as opposed to resisting it.”
Q1. You have 20+ years experience in product and engineering leadership. What are the main lessons you have learned?
Gokhan Uluderya: I think there are two very important lessons that I have learned throughout my career: first one is change is inevitable and is a constant in our lives and second one is that learning is a lifelong process. The most successful people that I have worked with and the most successful organizations I have been a part of have been the ones that embrace learning and change as opposed to resisting it. They had one thing in common and that was growth mindset: constantly learning from others, from one’s mistakes, open to change even if it went against the grain, and using these learnings to reinvent, transform, and innovate.
This is particularly important in technology sector because we are making progress at break-neck speed and we are surrounded by brilliant minds that are innovative, dedicated, and very passionate to make a difference in our lives and leave a mark in history. We are not only inventing new technology but also redefining how we live our lives, how we run our businesses, how we socialize with other people, and how we interact with the nature and the machine. Where there is constant change, there is also constant learning. This is true for organizations as well; small and large. Our organizations are constantly evolving. The organizational norms and behaviors are changing. Technological advancements are also impacting how organizations operate.
So, one of the biggest lessons for me as product and engineering leader is that we all need to continuously learn and reinvent ourselves, our organizations, and processes. Experimenting, making mistakes, failing are some of the most effective ways to learn and innovate. We should encourage these behaviors within our teams and constantly remind ourselves that very impactful things happen with small and consistent forward progress.
Q2. What is the main impact on customers’ businesses of all the transformations and disruptions that we have seen in the last 25 yrs?
Gokhan Uluderya: The pace at which disruptive transformations are happening has been accelerating exponentially in the last 25 years.
Think about it… When I started my career, mainframe-based architectures were still prominent. We went from mainframe-based architectures to SOA then SaaS, followed by the Cloud revolution. Mobile computing, IoT and edge computing brought amazing capabilities and experiences to the fingertips of every user and almost every device imaginable.
The cloud computing model enabled the builders and innovators in a fundamentally different way. The powerful compute cluster that would take 6-12 months to procure and provision in a classic enterprise environment with a very hefty price tag was now available in the cloud with a few clicks for a few hours of use for any developer. These transformations enabled big data and analytics workloads moving to the cloud and led to AI/ML being done at very large scale thanks to the scale and flexibility cloud provides. All these brought us to the age of generative AI and we are now moving fast towards artificial general intelligence.
The advancements in AI are changing the entire technology and business landscape in fundamental ways. It is changing the way we consume technology and the way we live our lives. AI-driven automation, decisioning, hyper-personalization complemented by natural-language understanding and generative AI is moving the effectiveness of technology to a whole new level.
These transformations are very exciting, and they create a lot of new opportunities. Consumers feel the impact from to shopping to dining, in communications and social interactions, in health services they receive, and in the cars they drive. Businesses feel the impact in all business functions; sales, marketing, customer service, supply chain and logistics, hiring practices are all being re-imagined with AI.
Every business is trying to adapt to and adopt the innovation coming down the pipe and fear-of-missing-out is driving them to make big decisions and investments faster than they are used to.
Q3. What are the challenges, the lessons learned, and the recommendations for both customers and tech companies that are going through these transformations and disruptions?
Gokhan Uluderya: In this kind of a rapid-changing landscape, it is hard to keep up with the change on all fronts. It is easy to make mistakes and some mistakes can be costly. Winners and losers can change very quickly. As technology vendors, we must keep in mind that our duty to our customers is not only to bring them new capabilities and innovation but also do it in a responsible, trusted way and help them along their journeys as they try to adapt and transform.
One of the key lessons learned and our recommendations to our customers is to play the long game, keep calm, and make steady progress. It is easy to go after the shiny objects and hype and fall victim to FOMO. We advise our customers to have a long-term view and strategy and evaluate the incoming disruptions carefully through that lens. It is generally a great strategy to adopt and transform in a spiral pattern: have a strong business value hypothesis, start with a small pilot, validate technology and solution, prove business value and adoption before moving to the next cycle up in the spiral.
Be agile but not random. I have seen a lot of organizations confuse agility with having no strategy, vision, or plan. Some think it is a process thing like “scrum is agile, but waterfall is not”. Agile doesn’t mean “go where the wind blows on any given day”. Agility gives the best outcomes when an organization has done enough homework to develop a long-term conviction and a plan and is able to make the right changes to their roadmap based on their experiments and learnings throughout that journey.
It also makes a huge difference to have a trusted partner that is a companion to you on your journey rather than just a vendor. Look for those trusted partners that have a vested interest in making you successful with your journey rather than vendors who are motivated mainly by selling you more.
Q4. You are the Head of Product at InterSystems. What are your responsibilities?
Gokhan Uluderya: As Head of Product for Data Platforms at InterSystems, I am responsible for leading the teams that drive our innovation in Data and AI technology space. Our Data and AI technologies power the most mission critical workloads in healthcare, finance, CPG, and supply chain. In our portfolio, we have several products that are industry-leading solutions. InterSystems IRIS and IRIS for Health lead in the multi-model, translytical data platform space. Our Health Connect product line is a leading interoperability platform solving some of the most complex integration and interoperability problems in healthcare space. We also provide several Cloud Data Servies such as IntegratedML, Cloud SQL, Cloud Document as SaaS offers. InterSystems Data Fabric Studio is one of our more recent innovations and it enables our customers to solve the enterprise Data & AI problem by building smart data & AI fabrics.
My team builds our core capabilities in this space and helps our customers be successful with our data & AI products and with their data & AI strategy.
Q5. What is your experience in working with data, AI, and analytics?
Gokhan Uluderya: Before joining InterSystems, I spent close to 5 years at Salesforce and 14 years at Microsoft at various roles related to data, AI, and analytics. At Microsoft, I was one of the founding members of Azure Machine Learning services as part of the product leadership team starting in 2015. During my tenure at Azure Machine Learning, I also received my data science certification from Harvard University to deepen my knowledge of the theory of AI and ML.
During my time at Salesforce, I was the VP of Product and GM for Marketing Cloud Personalization and VP of Product for Commerce Cloud leading hi-scale commerce services and Commerce Einstein teams. We used AI/ML to build hyper-personalized experiences for consumers and AI/ML-driven decisioning and automation for back-office applications.
Now in my role at InterSystems, I am building data & AI technologies that solve very important problems in healthcare and life sciences, finance, CPG, and supply chain management.
Q6. InterSystems has been recognized as a Leader in The Forrester Wave™: Translytical Data Platforms, Q4 2024. What is a Translytical Data Platform? And why does it matter?
Gokhan Uluderya: A translytical data platform supports transactional and analytical workloads in real-time from a single system. It also provides support for multi-model data from the same system for structured, semi-structured, unstructured data. InterSystems IRIS is a leading translytical data platform that reduces architectural complexity and provides unparalleled performance and scale for operational and analytical workloads delivering real-time processing. We currently have customers that go up to 1.6 Billion transactions per second on a single deployment in the lab environment and more than 150M transactions per second in a single deployment of IRIS in production. Many data platforms may claim translytical support by bolting multiple technologies together, but InterSystems IRIS has a unique, differentiated architecture that makes it translytical natively in one system and for that reason it provides an unparalleled price-performance value in addition to performance, scale, and reliability.
Q7. Your data platform supports multi-model natively. What does it mean and what are the main benefits?
Gokhan Uluderya: Multi-modelity is the ability to support structured, unstructured, semi-structured data models from the same system. These data models can be relational, document, columnar, vector, graph, time-series etc. There are many database engines in the marketplace today that are specialized in one or many of these data models. What we mean by supporting multi-model natively is the fact that our data platform has a unique and differentiated architecture that provides these data models from one unified common data plane. Different data models are represented in a unified object-based architecture and easily projected into different data models.
This is very significant in the world we live in today because to be able to get that customer 360, business 360, or data 360 view you need to bring many different data types together. AI and Generative AI capabilities made this trait even more important because AI can consume all of the data together to make decisions. For example, to provide AI-assisted experience for a patient, you need to have a complete view of their interactions such as the relational data in an EMR, handwritten note from a doctor, X-ray images or summary document from another visit, or voice recording from the last encounter.
Being able to manage all these data on InterSystems IRIS natively and being able to use them for decisioning is a very powerful proposition for our customers.
Q8. What plans and vision do you have for the future?
Gokhan Uluderya: The AI wave we are going through right now is a game changer. This is not because AI is a new concept or technology; it is because Generative AI and particularly ChatGPT democratized AI in a very short period. AI-driven experiences were already part of many of our experiences, however progress in natural language understanding, generative AI, and AI-driven reasoning coming together in ChatGPT suddenly brought AI conversation to our kitchen tables.
Our vision is that this democratization and proliferation of AI-driven experiences will continue at a very high speed, and it will continue to revolutionize all aspects of life and business. AI-based experiences, automation, reasoning and decisioning, “artificially intelligent” machines will become ubiquitous. I also believe this will remind everyone once again how invaluable trusted, private data is and what kind of a differentiator it is; especially in the AI-driven world.
To get the right outcome from any AI system, to make any important decision, we need to feed it with clean, comprehensive, and more importantly trusted data. As a data and AI technology company, our plan is to empower our customers to build data & AI solutions in a trusted, reliable, cost-effective way. We will give our customers the data & AI fabric that allows them to build and capitalize on their data assets. We will optimize and automate their data & AI workflows using AI. And most importantly, we will infuse AI-driven experiences into all our solutions and applications in healthcare, finance, CPG, and supply chain.
Qx. Anything else you wish to add?
Gokhan Uluderya: I would like to encourage our readers to take a look at InterSystems IntelliCare. InterSystems IntelliCare is EHR reimagined using the power of AI and GenAI. It empowers clinicians, enhances patient experiences, elevates business operations, and minimizes resource utilization using AI and GenAI.
It is a great example of our vision in action: InterSystems data and AI technologies revolutionizing our experiences in healthcare which we all deeply care about.
…………………………………………
Gokhan Uluderya , Head of Product, InterSystems Data Platforms.
Executive leader with 20+ years of product and engineering leadership experience at leading software and technology companies (Microsoft, Salesforce, and Verizon) delivering multiple 0-1 products and services that have grown to be highly-successful businesses.
Experienced in designing and developing large-scale, cloud services (SaaS, PaaS), on-premises server products, and enterprise applications driving product strategy, roadmap, and delivery managing geo-distributed, cross-disciplinary teams (product, engineering, research, customer success, design, UA, and enablement)
A technical innovator in Data, AI/ML, Personalization space and a business leader driving a business unit as general manager responsible for P&L and C-Suite relationships.
The Forrester Wave™: Translytical Data Platforms, Q4 2024
Nov 6, 2024 — This report shows how each provider measures up and helps data and technology professionals select the right one for their needs.
Related Interviews
On Applying Data Fabrics Across Industries. Q&A with Joe Lichtenberg.
On Data Fabric and Data Mesh. Q&A with Jeffrey Fried
………………………..
“Cloud is a tool, not a destination. Hybrid is the strategy that delivers both control and agility.”
Q1. What is your role and current projects at Exasol?
Mathias Golombek: As CTO of Exasol, I oversee the technical direction of our high-performance Analytics Engine, ensuring it delivers speed, scalability, and cost efficiency across on-premises, hybrid, and cloud environments. My role involves driving innovation in query performance, self-tuning optimizations, and seamless data integration to help organizations maximize the value of their analytics.
Right now, we’re focusing on:
- Enhancing hybrid data integration, making it easier for companies to run analytics across on-prem and hybrid environments without performance trade-offs.
- Optimizing our query execution engine, improving parallel processing, indexing strategies, and workload balancing to ensure consistently fast performance.
- Expanding AI/ML capabilities, enabling advanced analytics workloads directly within the database without the need for additional infrastructure.
- Improving cost efficiency, refining storage and memory management to reduce operational costs while maintaining top-tier performance.
These initiatives ensure Exasol remains the most powerful, flexible, and cost-effective analytics solution for data-driven organizations.
Q2. Choosing the right infrastructure is a strategic decision that will impact an organization for years. What practical tips can you offer in this area?
Mathias Golombek: The key to making the right infrastructure choice is understanding workload requirements, regulatory constraints, and total cost of ownership (TCO). Organizations should ask:
- Performance Needs: If real-time analytics and low-latency queries are critical, an in-memory Analytics Engine like Exasol can provide superior performance than other databases.
- Data Governance & Compliance: If strict data residency and compliance laws apply (e.g., GDPR, HIPAA), an on-premises or hybrid approach may be necessary.
- Cost Predictability: Cloud costs can spiral if not managed effectively. Organizations should model workloads and compare TCO across on-prem, hybrid, and cloud options.
- Scalability & Integration: Consider the need for seamless integration with existing tools and the ability to scale without costly re-architecture.
- Future-Proofing: The landscape evolves rapidly—opting for an infrastructure that supports flexibility (on-prem, cloud, hybrid) ensures long-term adaptability.
At Exasol, we see organizations increasingly favoring a hybrid-first approach, leveraging on-prem for mission-critical workloads while optimizing cloud usage for elasticity and burst processing.
Q3. Let’s talk about migrating to the Cloud: Why, How and What Makes Sense?
Mathias Golombek: Cloud migration is often driven by scalability, elasticity, and ease of management. However, it’s not a one-size-fits-all solution. The key considerations are:
- Why migrate? Organizations move to the cloud for agility, operational simplification, and dynamic scaling. However, performance-sensitive workloads may not see cost or speed benefits.
- How to migrate? A phased approach works best—starting with non-critical workloads, leveraging hybrid setups, and optimizing data architecture to prevent unnecessary cloud egress costs.
- What makes sense? A cloud-smart strategy rather than a cloud-first mandate. Many organizations are now repatriating workloads due to unpredictable costs and performance inefficiencies. Workloads requiring low latency, predictable costs, and high security often perform best on-prem or in a hybrid model.
Exasol supports flexible deployment, allowing organizations to run the same high-performance analytics across on-prem, hybrid, or cloud environments—giving them the ability to adjust strategies as needed.
Q4. On the contrary, what is the biggest advantage of on-premises computing?
Mathias Golombek: The biggest advantage of on-premises computing is predictability—in cost, performance, and security.
- Performance Optimization: On-prem allows full control over hardware and resource allocation, minimizing latency and delivering consistent high-speed analytics.
- Cost Efficiency at Scale: While cloud pricing is attractive for small workloads, long-term costs often escalate due to unpredictable storage, compute, and egress fees. A well-optimized on-prem solution has a lower total cost of ownership (TCO) over time.
- Data Control & Compliance: Industries like healthcare, finance, and government require stringent data sovereignty, regulatory compliance, and security—challenges that cloud providers can’t always meet.
- Minimal Vendor Lock-in: Cloud providers have proprietary ecosystems that can make data migration complex and costly. On-premises solutions allow full control over data access, storage, and portability.
For organizations running high-performance analytics on large datasets, Exasol’s in-memory Analytics Engine on-premises consistently outperforms cloud alternatives while maintaining cost predictability and compliance advantages.
Q5 According to a Barclay’s CIO survey, 83% of enterprise CIOs planned to repatriate at least some workloads in 2024. What are the reasons why companies are choosing to bring their data in-house?
Mathias Golombek: We’ve seen this shift in our own customer base. The primary drivers for workload repatriation include:
- Cost unpredictability: Cloud egress fees and unpredictable pricing models have made on-prem/hybrid more attractive for long-term analytics workloads.
- Security & control: The rise of AI and sensitive data analytics has made many organizations reconsider who controls their data and how it’s stored, processed, and accessed.
- Performance bottlenecks: Latency and performance inconsistencies in shared cloud environments make real-time analytics and high-concurrency workloads challenging.
- Regulatory compliance: Industries like banking, healthcare, and telecom face increasing data sovereignty and privacy regulations, making on-premises or hybrid solutions more viable.
- Many of these shifts and their implications have been widely discussed: hybrid strategies—where compute happens on-prem while scalability is extended via cloud—are now the preferred model.
Exasol excels in hybrid environments by providing high-speed analytics while ensuring full control over data location and processing.
Q6. According to the latest forecast by Gartner, 90% of organizations will adopt a hybrid cloud approach through 2027. What is your take on this?
Mathias Golombek: The hybrid cloud model is not just a transition phase—it’s the future of enterprise IT.
- Best of both worlds: Companies are realizing that on-prem is critical for cost efficiency, performance, and compliance, while cloud provides agility and elasticity.
- Cloud is not always cheaper: Many organizations initially moved to the cloud expecting lower costs but are now balancing workloads between cloud and on-prem to optimize spend.
- Interoperability is key: Businesses need infrastructure that integrates seamlessly across on-prem, private cloud, and public cloud without vendor lock-in.
At Exasol, we design for hybrid-first strategies—enabling organizations to scale analytics seamlessly across on-prem, hybrid, and cloud without sacrificing speed or cost efficiency.
The key takeaway? Cloud is a tool, not a destination. Hybrid is the strategy that delivers both control and agility.
Q7. In your opinion what are hybrid cloud benefits?
Mathias Golombek: A hybrid cloud strategy combines the best aspects of on-premises and cloud computing, offering organizations flexibility, performance optimization, and cost efficiency while maintaining control over security and compliance. The key benefits include:
- Optimized Workload Placement: Certain workloads—such as real-time analytics and high-concurrency queries—perform better when executed on-premises due to low-latency in-memory processing and predictable performance. Cloud resources can be leveraged for burst capacity, external data ingestion, or long-term storage.
- Cost Efficiency & Resource Utilization: High-performance engines like Exasol can minimize compute overhead in an on-prem deployment while still integrating with cloud object storage for cost-effective data retention.
- Data Sovereignty & Compliance: Many industries—healthcare, finance, public sector, and telecommunications—require strict data residency controls. Hybrid cloud enables organizations to process and store sensitive data on-prem while leveraging cloud services for non-sensitive workloads.
- Scalability & Elasticity: Organizations can dynamically scale resources by leveraging the cloud for compute-heavy tasks (such as machine learning inference) while keeping mission-critical workloads running on-prem for predictable performance.
At Exasol, we optimize for hybrid deployments, ensuring seamless data virtualization, query federation, and cross-platform analytics without performance degradation.
Q8. The most common hybrid cloud example is to use public cloud with private cloud services and on-premises infrastructure. What is your take on this?
Mathias Golombek: A hybrid cloud model that combines public cloud, private cloud, and on-premises infrastructure is increasingly becoming the standard. However, the key challenge is not just deployment but ensuring seamless workload portability and data interoperability across environments.
- Latency & Performance Considerations – High-performance analytics workloads often require low-latency query execution, which is best achieved with in-memory, on-premises infrastructure or high-performance private cloud deployments rather than public cloud services optimized for storage rather than compute.
- Data Gravity & Egress Costs – Moving data between environments introduces latency penalties and unpredictable cloud egress costs. Organizations must optimize data locality and workload placement to minimize transfer inefficiencies.
- Security & Compliance – Private cloud helps enforce data sovereignty and regulatory mandates, but integration with public cloud analytics tools often leads to security trade-offs and additional access control requirements.
- Cross-Platform Query Execution – A hybrid approach only works effectively when databases support federated query execution, virtualization, and schema bridging, ensuring that data silos are avoided, and workloads can scale efficiently across environments.
I see hybrid architectures not as a static setup but as an evolving, workload-aware strategy. Exasol’s Analytics Engine enables high-speed analytics across hybrid infrastructures by minimizing query latency, optimizing data locality, and integrating seamlessly with cloud and on-prem ecosystems—allowing organizations to maximize performance without unnecessary complexity.
Q9. What are the requirements to build and deploy Generative AI workloads?
Mathias Golombek: Deploying Generative AI (GenAI) workloads effectively requires a combination of high-performance compute, scalable storage, optimized data pipelines, and inference acceleration. The key requirements include:
High-Performance Compute Infrastructure
- Parallel Processing & MPP Architectures: Training and running large foundation models require distributed computing frameworks to optimize vectorized execution and parallel workloads.
- GPU & TPU Acceleration: Many transformer-based models rely on GPU/TPU acceleration for efficient matrix multiplications and tensor operations.
Scalable & High-Speed Storage
- Hybrid & Multi-Tiered Storage: Storing training datasets in a combination of on-prem NVMe storage (for high-speed access) and cloud object storage is a common approach.
- Data Lake Integration: Exasol’s query engine can be used to process structured and semi-structured data efficiently, ensuring high-throughput data preparation for AI pipelines.
Optimized Data Management & Feature Engineering
- Federated Data Access: GenAI models require diverse datasets—ranging from structured enterprise data to unstructured text, images, and videos. Hybrid environments must support fast ETL processes and federated queries across multiple sources.
- Vectorized Execution & Feature Store: Efficient feature engineering requires databases that support vectorized processing, indexing, and real-time transformations, with integration options for feature storage and retrieval in AI/ML workflows.
Inference Optimization & Model Deployment
- Inference Optimization & Data Access: AI workloads require efficient data retrieval and transformation pipelines. Exasol enables near real-time analytics and feature engineering for AI models while integrating with external ML platforms for model training and inference.
- Real-Time AI Integration: Using high-speed analytical databases like Exasol ensures that GenAI models can query and process real-time data without performance bottlenecks.
Security, Compliance, & Governance
- Data Sovereignty & Compliance Controls: Many AI workloads process sensitive PII data, requiring on-prem data governance while allowing cloud-based AI training.
- RBAC & Secure AI Pipelines: Implementing role-based access control (RBAC), model versioning, and explainability frameworks ensures AI transparency and compliance with industry standards.
How does this work in practice? For example, with Exasol, users can integrate with LLMs in 3 ways:
- Exasol In-database LLM Deployment:
Download your chosen language model into Exasol’s internal file system (BucketFS) and access it via User Defined Functions (UDFs). This method guarantees that your data, queries, and prompts remain securely within your environment, minimizing exposure to external networks.
- Connect to locally hosted LLM:
Integrate with LM Studio and other language model services managed within your own network and infrastructure for a balance of security and flexibility.
- API-Based Integration:
Connect directly to external language model APIs using UDFs. This option provides rapid access to the latest models without the need for local deployment, offering flexibility and speed.
We focus on accelerating AI-driven analytics by providing low-latency, high-performance query processing, ensuring efficient data preparation, real-time feature engineering, and on-premises and hybrid AI deployments.
Qx. Anything else you wish to add?
Mathias Golombek: As organizations continue to evolve their hybrid and AI-driven analytics strategies, the focus should be on:
- Workload-specific infrastructure choices rather than forcing cloud adoption where it doesn’t provide cost or performance benefits.
- Optimizing structured data processing to support AI-driven insights and decision-making while ensuring seamless integration with external unstructured data sources.
- Minimizing operational complexity by leveraging self-tuning, high-performance analytics engines that seamlessly integrate across on-prem, cloud, and hybrid environments.
At Exasol, we are committed to pushing the boundaries of analytics performance, ensuring organizations can extract real-time insights from massive datasets while optimizing cost, scalability, and security.
………………………………………….
Mathias Golombek
Mathias Golombek is the Chief Technology Officer (CTO) of Exasol. He joined the company as a software developer in 2004 after studying computer science with a heavy focus on databases, distributed systems, software development processes, and genetic algorithms. By 2005, he was responsible for the Database Optimizer team and in 2007 he became Head of Research & Development. In 2014, Mathias was appointed CTO. In this role, he is responsible for product development, product management, operations, support, and technical consulting.
…………………………………………………………..
Related Posts
On Generative AI. Q&A with Bill Franks
………………………………………………………….
On Generative AI and Consumer Marketing. Interview with Ziv Fridfertig and Madhukar Kumar.
“In many ways the Large Language Models (LLMs) have become commoditized, and the true differentiator is now the substrate of intelligence – which is data. Companies that take advantage of their data to drive richer and better experiences for their customers with AI are going to be the clear winners. “
Q1. How can generative AI boost consumer marketing?
MK: Gen AI helps marketers be more agile in fast-changing markets and do more with fewer resources. It’s about scaling your creativity, personalizing the customer experience, and optimizing campaign strategy and execution. It’s also about automating repetitive tasks – getting more of the grunt work done extremely fast. With gen AI, everything from ads to social media to email campaigns can also be thoroughly personalized and targeted. AI-guided chatbots and virtual assistants can learn from customer preferences and behavior patterns to make better recommendations and ultimately improve the customer experience. And by analyzing trends and making predictions, gen AI can play a role in audience segmentation and strategy – even modifying campaigns dynamically based on real-time feedback and data. Finally, AI tools can help marketers make data-driven decisions by analyzing data without getting an entire team of data analysts involved.
Q2. A recent McKinsey report estimates that gen AI could contribute up to $4.4 trillion in annual global productivity. What is your take on this?
MK: I believe it. Using consumer marketing as an example, the cost savings and productivity gains are real. Gen AI is already automating some of consumer marketing’s more routine and time-intensive tasks, giving creative teams more time to focus on strategy. For example, gen AI can do scalable A/B testing to test what resonates best, design concepts and layouts, or localize content for different regions and languages. In all of these ways, gen AI is a powerful tool for consumer marketers who seek to do more with less. Similar gains are happening in every industry, so you can easily see how McKinsey has reached such bullish conclusions.
Q3. What are the ways consumer companies can create value with gen AI?
MK: In many ways the Large Language Models (LLMs) have become commoditized, and the true differentiator is now the substrate of intelligence – which is data. Companies that take advantage of their data to drive richer and better experiences for their customers with AI are going to be the clear winners.
One great example of a company realizing these benefits is Skai. Skai’s AI-powered platform offers consumer and market insights and omni-channel media activation services. As with so many of our martech and adtech customers, Skai’s integration of gen AI helps them deliver more personalized, efficient, and innovative marketing solutions.
Skai was one of SingleStore’s first customers that built a multi-tenant service on SingleStore. Multi-tenant is now the most common deployment among all of our martech/adtech customers. Our partners in this deployment are AWS and also Twingo, who helped Skai deploy our solution and push the technology to the limits for optimal performance.
Q4. How does Skai use gen AI? And how does Skai use SingleStore?
ZF: We use gen AI in many of the ways Madhukar described above. It’s about eliminating the friction inherent in “walled garden media” – digital advertising and media ecosystems where the platform owner controls and restricts access to the data, content, and audience interactions within their environment. Our machine-learning algorithms and proprietary NLP break down those barriers, enabling companies to listen, predict and keep pace with the consumer journey.
We use SingleStore as our foundational database for real-time analytics and we use SingleStore Pipelines to ingest the massive amount of data that we require. SingleStore has saved our developers a lot of time while getting our users the real-time insights they need. Though we started with an on-prem deployment, we recently moved to the cloud with AWS, which has enabled us to more efficiently utilize our hardware and maximize our performance.
Q5. What are the main benefits for Skai’s clients? What are the main benefits of moving to what you call an omnichannel?
ZF: Our omnichannel advertising platform is the ultimate outcome of us listening to our client needs. We’ve been around for a while and learned that clients have their own systems and methods for marketing operations and deal differently with publishers, but there was a common ground – it’s expensive, confusing and exhausting with the fast pace of change.
Today our clients see the value of working with a single platform over multiple retailers.
It allows them much more visibility and control. The language gap is reduced. They can get insights faster, quickly identify and react to low-performing campaigns, save time with bulk operations and maximize their budget.
The platform includes smart customized widgets, real-time analysis, advanced targeting and automated actions. With this toolset, campaign managers can easily oversee their campaigns across 100 publishers. Our AI-based features inform a strategic dashboard that gives marketing directors with actionable insights on forecasting and trends.
Q6. What data challenges do you face at Skai?
ZF: Skai has quite a unique setup, with data ingestion by 30,000 Kafka pipelines across tens of clusters, and we must cater to a high concurrency of client requests, made of 80 tables joined in a single query. That tends to push database technology to its limits.
And huge amounts of fast-moving data also creates hardware challenges. Hardware is a big investment so it can be difficult to keep pace with evolving data demands and processing needs. And administering thousands of servers isn’t cheap either.
Moving this system to AWS cloud improved our infrastructure in a few matters.
With EBS performance we migrated most of our in-memory data to storage, the outcome was reduction of hardware demands by half and improving DB performance.
With EC2 Multi-AZ we built an advanced setup to endure hardware failures without downtime, and managed to make our daily backups on S3 redundant.
Q7. You have built a multi-tenant service on SingleStore. Could you please tell us a bit about it? Why did you choose a multi-tenant service?
ZF: In a multi-tenant architecture, a single instance of a software application or platform is shared among multiple customers or users, known as tenants. Each tenant’s data is isolated and remains invisible to others, but they share the same infrastructure, resources, and core application services.
With SingleStore, we managed to achieve unmatched query performance which is impossible to get with single-tenant or inefficiently too expensive. It is cost effective, easier to manage, it is more resilient, data scale becomes a non-issue and we minimized our maintenance downtimes.
Ultimately, being multi-tenant makes a bigger impact for our clients. Deployments are easier, faster, and give us more horsepower for peak or ad-hoc loads, which keeps our user interface consistently responsive.
Q8. Why did you choose SingleStore? What are the main benefits?
ZF: Speed, scale and simplicity. SingleStore had the best query performance for our platform’s main use case. In just milliseconds, it can aggregate millions of metadata rows with billions of performance events. It can handle more than 80 tables in a single query with high concurrency. We love that the data lands in near real-time from Kafka to SingleStore, it was a hard requirement at the beginning of our journey 6 years ago.
Scalability was also a key. Big clients have big data and we need to grow along with them, with amazing query performance and zero downtime.
For simplicity, I would note the single dialect. MySQL compliance means that no special treatment was needed for building queries. In SingleStore, we don’t have any issues with inconsistency in querying.
Qx. Anything else you wish to add?
ZF: On a personal note, I project a good future for SingleStore and Skai. Both companies are attentive to client needs and success and share a vision for a centralized eco-system for data processing. With this great partnership, I’m excited for the upcoming capabilities in the cloud.
__________________________________________________________________________
Madhukar Kumar, chief marketing officer, SingleStore
Madhukar is a developer turned growth marketer and an expert in product-led growth (PLG) with more than 18 years of experience leading product management and marketing teams. He has successfully implemented PLG at Nutanix, Redis and DevRev, and is a guest lecturer at Duke University on PLG and new product development.
Ziv Fridfertig, Data Infra Manager, Skai
Ziv is a Data Infrastructure Manager with extensive experience in the ad-tech industry. His work focuses on managing and optimizing complex data solutions, building scalable data pipelines, and implementing real-time streaming systems that support high-performance applications.
Core Expertise:
- Leading and mentoring teams to achieve technical and organizational goals.
- Managing diverse data ecosystems, including RDBMS, NoSQL, Big Data, and distributed systems.
- Driving successful cloud migrations to improve scalability and efficiency.
- Designing and implementing architecture for microservices and cloud-based solutions.
With a strong foundation and experience in data management, he is passionate about delivering solutions that create tangible business value and drive long-term impact.
Resources
AWS re:Invent,December 2-6, 2024: Agenda
Related Interviews
– On AI Factory and Generative AI. Interview with Ashok Reddy. ODBMS Industry Watch. March 29, 2024
– On the Future of AI. Interview with Raj Verma, ODBMS Industry Watch, January 5, 2024
………………………….
“ Data silos dramatically slow down digital transformation”
Q1. Data protection, data sovereignty and data privacy: how do they relate to each other?
Paul Speciale: Data protection in its strictest sense is the best practice of protecting sensitive information from corruption or data loss. Key foci lie on privacy and confidentiality of data. The safeguarding of the data goes along with ensuring data availability as well as compliance with legal and regulatory requirements. In contrast, data sovereignty empowers countries to protect the privacy and security of data that is physically located, stored, processed, and used within their borders. Data sovereignty is part of a larger goal: digital sovereignty. Digital sovereignty provides control over digital destiny – a complete autonomy that encompasses the entire end-to-end ecosystem and infrastructure. Data privacy and data sovereignty are closely related. Data privacy is primarily focused on the control of persons over their respective data. This includes the ability to make a decision about how organizations collect, store, and use personal data. Data privacy regulations are mainly determined by data residency and provenance.
Q2. Are there any differences between data sovereignty, data residency, and data localisation?
Paul Speciale: As the name implies, data residency refers to where data is actually stored. Data residency is concerned with the exact geographical location of data. As mentioned above, data sovereignty (or data independency) has its focus on control over data based on the legal framework, the jurisdiction of the data storage and processing within a country. Whereas data localization identifies the precise geographic location of data: at its heart is the storing and processing of data in the same country where it was originally created. Namely the EU General Data Protection Regulation (GDPR) requires this.
Q3. Data sovereignty promises to break up data silos and enable the usage of data across different stakeholders that could not be shared with others so far. What is your take on this?
Paul Speciale: Data silos dramatically slow down digital transformation. However, they are no longer an obstacle if the data of an organization is to be brought together and optimized for analysis purposes. Modern cloud technology has been enhanced in a way to make such centralisation possible. Data sovereignty in the context of the cloud ensures that individuals and entities have greater visibility and thus control over their data, deciding how it is used and shared beyond the confines of silos.
Q4. What is data sovereignty in the Cloud?
Paul Speciale: Capabilities with the overall goal to ensure data sovereignty in the cloud are needed, and some cloud providers have this on their radar. However, to navigate beyond the complexity of obligations stemming from regulations, and in order to achieve true data sovereignty and data independence, a tailored approach needs to be in place that shows alternatives to relying entirely on the public cloud. Scality offers cloud-like storage that is hosted on-premise, with the same agility and standard S3 APIbut with more precise location control over data.
Q5. AI, LLM Data Privacy and Data Sovereignty: What are the challenges here?
Paul Speciale: AI is moving quickly. In the hurry to embrace AI and turn it into a part of their respective operations, enterprises need to ensure that they have critical areas on their radar, with the overall goal of not putting themselves out of compliance accidentally. Within due diligence, organizations can minimize AI’s inherent risks by addressing the right questions. Among these are: Is the cloud provider of choice suitable to help guarantee data sovereignty, or should AI Data lakes be built and managed on-premises for improved control? This can be ascertained by way of checking certificates. An additional question: where is the basic data for AI applications stored – and who is able to access, and monitor it? The role of Large Language Models (LLM) in AI is significant. Companies that are considering integrating LLMs into their workflows are well advised to include a risk analysis for their specific use cases into their strategies.
Q6. EU new regulations to protect data (NIS2 and DORA) come into effect in October 2024 (NIS2) and in January 2025 (DORA). Can you briefly explain what NIS2 and DORA stand for and what they regulate?
Paul Speciale: TheNetwork and Information Security Directive 2.0 (NIS-2) is the EU’s new, mandatory cybersecurity regulation. It will be implemented in October 2024. Organizations in certain industries (namely financial services, energy, transport, as well as digital services sectors) must proactively take appropriate cybersecurity measures. They also must instantly report significant incidents. The key goals of NIS-2 are to strengthen overall resilience to cyber-attacks, and to help improve responsiveness in the event of security issues. NIS-2 is setting requirements for risk management as well as incident reporting. DORA (Digital Operational Resilience Act), which will come into force in January 2025, aims to ensure resilience to cyber-attacks for the financial sector. It follows the guidelines of the European Banking Authority.
Q7. What are the implications of such data protection regulations in practice?
Paul Speciale: Initially, companies should check whether and how they will be affected by these new regulations. In a next step, IT teams should analyze how they can enhance their existing cyber security measures to make sure they are aligned with the new legal requirements. A central aspect for all companies affected is to implement stronger cyber-resiliency measures on their data, which can start with the deployment of immutable data storage for backups to ensure that data can be recovered in the event of a breach.
Q8. What if an organization relies on content distribution or content ingestion from a global perspective? What are the challenges an organization faces in this case?
Paul Speciale: EU organizations with requirements for cross-border data flows will face challenges due to these new data sovereignty requirements, mainly in limitations on how data is transferred. For instance, organizations handling critical infrastructure data may face restrictions on exporting data outside the EU. Solutions might require the need to set up separate EU based data centers or comply with strict mechanisms for these data transfers. This could complicate operations and lead to inefficiencies.
Q9. In your opinion, what is the best approach to store data to deal with compliance regulations?
Paul Speciale: To effectively deal with compliance regulations, especially in light of frameworks like NIS2, DORA, GDPR, and other global standards, organizations may consider adopting hybrid-cloud strategies, establishing regional data centers, implementing strong encryption (at-rest and in-transit), enforcing data retention policies and ensuring they have immutable backups and disaster recovery policies.
Q10. What is Scality’s S3 object storage and how can it help here?
Paul Speciale: Scality is at the forefront of the S3 Compatible Storage trend with multiple commercial products and open-source projects. Scality RING offers an object storage solution with a native and comprehensive S3 interface. Scality RING is the first Amazon S3-compatible object storage for enterprise applications with secure multi-tenancy, multi-level cyber-resilience, data location control with support for single and multi-geo deployments.
……………………
Paul Speciale, CMO, Scality
Paul Speciale is currently the Chief Marketing Officer at Scality. Prior to this, Paul held various leadership roles in companies such as Appcara, Amplidata, and Savvis, focusing on cloud computing and storage technologies. With a background in technology consulting and database architecture, Paul has a strong foundation in the IT industry.
“ The AI Factory is a revolutionary concept aimed at streamlining the creation and deployment of AI models within organizations. It serves as a centralized hub where data scientists, engineers, and domain experts collaborate to develop, train, and deploy AI solutions at scale.“
Q1. You joined KX as Chief Executive Officer in August 2022 (*). How has the database and the AI market developed since then?
Ashok Reddy: Since taking on the role of Chief Executive Officer at KX in August 2022, I have observed significant transformations within the database and AI markets, underscored by burgeoning investments and innovations across several key segments:
Non-relational Databases (NRDBMSs): These databases have experienced notable growth, driven by the demand for flexible, scalable data management systems that accommodate the complex needs of modern, data-intensive applications.
Analytics and Business Intelligence Platforms: This segment has continued to expand rapidly, fueled by the increasing need for sophisticated analytical tools that can provide deeper insights into vast datasets, enabling more informed decision-making processes.
Data Science and AI Platforms: The emergence and integration of advanced data science and Generative AI (GenAI) technologies have propelled this sector forward, with organizations seeking powerful platforms that can drive AI-driven innovation and operational efficiency.
These industry segments have showcased annual growth rates ranging from 20 to 25%, highlighting a substantial shift towards technologies that are not only agile and scalable but also capable of underpinning advanced analytics and AI-driven applications.
Q2. It seems that securing a tangible return on investment from artificial intelligence (AI) is still a challenge. Do you agree? How can you ensure your AI has an ROI?
Ashok Reddy: Yes, achieving a tangible return on investment from artificial intelligence (AI) poses a challenge, but it’s not unattainable. AI, fundamentally a prediction technology as highlighted in “Power and Prediction” by Ajay Agrawal, Avi Goldfarb, and Joshua Gans, and “Competing in the Age of AI” by Karim R. Lakhani and Marco Iansiti, offers organizations the ability to make informed predictions based on extensive datasets. This capability is key to gaining a competitive advantage.
To ensure AI initiatives yield a tangible ROI, a strategic focus on leveraging prediction technology is crucial. This involves minimizing prediction costs, which can be achieved by reducing the incidence and impact of prediction errors and continually refining AI models to enhance their accuracy. By strategically lowering these costs, companies can boost operational efficiency, foster innovation, and elevate revenues.
Moreover, viewing AI’s predictive prowess as a strategic asset allows for the alignment of AI endeavors with specific business goals. Whether the aim is to streamline operational processes, improve customer experiences, or explore new market opportunities, the predictive capacity of AI can be effectively utilized to achieve concrete business results.
Q3. What is the AI Factory concept, and what is its significance?
Ashok Reddy: The AI Factory is a revolutionary concept aimed at streamlining the creation and deployment of AI models within organizations. It serves as a centralized hub where data scientists, engineers, and domain experts collaborate to develop, train, and deploy AI solutions at scale. The AI Factory is essential for organizations looking to leverage AI effectively across various business functions, enabling them to drive innovation, enhance productivity, and gain a competitive edge.
Q4. How does an AI Factory facilitate the utilization of a single GenAI model across multiple functions and tasks?
Ashok Reddy: An AI Factory empowers organizations to leverage a single GenAI model for a multitude of functions and tasks through systematic processes and automation. By centralizing model development and deployment, the AI Factory ensures consistency, scalability, and reusability of AI solutions. This enables organizations to streamline workflows, optimize resource allocation, and extract maximum value from their AI investments.
Q5. What are your tips for scaling AI in practice?
Ashok Reddy: Adopting an AI Factory approach will enable businesses to scale AI. In practice, this means changing how we think about the typical tasks our workforce is asked to complete and how we apply [AI] technology to support not only the completion of that task but the entire job motion itself. For example, if we can support paralegals to do the document review and preparation that a lawyer typically handles, we’re enabling that lawyer to dedicate more of their time to complex cases thereby revolutionizing the job processes for both sides.
By rethinking how we scale AI within the human workforce, we can empower professionals to pursue roles of greater strategic value, reduce time to market for AI solutions, and ensure consistency and quality across AI initiatives.
Q6. Let’s now look at GenAI. Several specialized companies offer so called GenAI “foundation models”– deep learning algorithms pre-trained on massive amounts of public data. How can enterprises take advantage of such foundation models?
Ashok Reddy: Foundation models provide a massive head start for enterprises, but their value lies in adaptation and thoughtful integration. Some examples of adoption of these models for organizations to consider are:
- Reducing Development Time and Costs: Instead of building complex AI models from scratch, enterprises can fine-tune foundation models for specific tasks, saving significant resources.
- Unlocking New Applications: The flexibility of foundation models (for text, image generation, etc.) enables creative applications, potentially disrupting existing workflows.
- Democratizing AI Access: Smaller businesses and those without in-house AI expertise can leverage pre-trained models via APIs or user-friendly platforms.
- The RAG Connection for Leveraging Private Data: Foundation models are the perfect ‘retrieval’ component of the retrieval-augmented generation (RAG) pipeline. Accessing knowledge at scale becomes easier.
However, to effectively deploy these strategies, enterprise organizations should consider putting a few guardrails in place, such as:
- Domain-Specific Fine-Tuning and Data: Introduce your own high-quality, domain-specific datasets to combat potential irrelevance or bias in the pre-trained model.
- Constraints and Sanity Checks: Embed rules derived from industry knowledge and common sense into your fine-tuned model to avoid unrealistic or undesirable outputs.
- Continuous Evaluation: Don’t treat a foundation model as static. Test regularly against real-world scenarios and adjust as needed
- Human-AI Collaboration: Emphasize explainability, critical evaluation of outputs, and maintaining human oversight. This is vital for trust and responsible deployment.
Q7. Three issues are very sensitive when talking about GenAI: quality and relevance of public data, ethics, and accountability. What is your take on this?
Ashok Reddy: I think these issues are important to consider as we, as a collective industry, work to create expanded capabilities in the area of GenAI.
When it comes to the quality and relevance of public data, the two most important considerations are bias amplification and domain mismatch. First, public datasets often contain biases. Enterprises need rigorous pre-processing and bias mitigation techniques to avoid harmful outputs. This should include work to identify and mitigate biases within the models during development and testing to make sure we’re approaching GenAI creation with a sense of fairness and inclusivity.
With regard to domain mismatch, it’s important to know that data used to train the foundation model may not align with your specific enterprise needs. Fine-tuning and supplementing with your own data is essential.
On the topic of ethics, we’re seeing instances of misinformation and deepfakes in our everyday consumption of news and social media. Organizations should ensure foundation models are not misused for malicious purposes by implementing safeguards and internal policies to guide responsible use. As a society, we need to consume with a discerning eye and make sure we hold creators accountable.
I believe that in the area of accountability, transparency, governance, and continuous monitoring of models are all critical issues to be addressed.
Where possible, aim for some level of transparency in your foundation model’s decisions. This builds trust and helps debug potential issues. Establish clear ownership and protocols for using GenAI within your enterprise, defining acceptable use cases and limitations.
And, as foundation models evolve, so too must your risk assessment and ethical considerations. Therefore, establishing a method for continuous monitoring is key.
To help accomplish oversight in these areas, it’s important that we adopt grounding strategies.
The first way to do this is to use your knowledge base as an anchor, supplementing public data with your curated knowledge base of reliable sources, financial reports, or industry-specific data. This helps the model gain a better understanding of your domain’s facts and principles.
Secondly, you should use RAG for verification. If your foundation model generates text, use a RAG model to cross-check its outputs against trusted knowledge sources, reducing the chance of spreading misinformation.
And finally, make sure to include explainability as a requirement. Prioritize techniques that give insight into why the model generates certain outputs. This helps spot issues and maintain a strong link to reality.
Q8. What staffing approach is required for operating an AI Factory? Is it about hiring new specialists or upskilling existing business and tech roles, or both?
Ashok Reddy: Operating a GenAI Factory requires a strategic approach to staffing that combines both hiring new specialists and upskilling existing talent. While recruiting individuals with expertise in AI, machine learning, and data science is crucial for driving innovation, it’s equally important to invest in upskilling existing business and tech roles. By fostering a culture of continuous learning and development, organizations can ensure that their workforce remains adaptable and proficient in leveraging advanced AI technologies effectively.
Q9. Harvard Business Review (**) reported that “Gartner research has identified five forces that will keep the pressure on executives to keep learning, testing, and investing in the GenAI technology: 1) Board and CEO expectations; 2) Customer expectations; 3) Employee expectations; 4) Regulatory expectations; and 5) Investor expectations.” What does it mean in practice?
Ashok Reddy: Boards and CEOs are looking to implement solutions that have clear data pipelines, experimentation processes and a focus on translating AI insights into actionable decisions, all attributes of a well-structured AI Factory for continuous innovation.
Customers want AI-enhanced experiences. Organizations can work to deploy AI from the Factory for personalized recommendations, intelligent chatbots, and streamlined processes. Constant evaluation and refinement are the keys to success.
When it comes to employees, using AI as a Co-pilot or Assistant can help upskill employees to collaborate with the AI tools developed in their AI Factory. This reduces mundane tasks and fosters a sense of empowerment.
When it comes to regulatory expectations, you can look to explainability and bias mitigation and work to design an AI Factory with transparency in mind. Incorporate tools and processes to explain AI outputs and proactively address potential biases in datasets and models.
Investors are going to look for an AI Factory to produce ROI. For this, organizations can demonstrate a clear link between their AI Factory investments and tangible business outcomes (cost savings, revenue growth, risk reduction), while also working to underscore their commitment to ethical and responsible AI use.
An AI Factory approach isn’t a magic solution but a strategic framework. Executives who develop a structured plan for building their AI Factory, one that is responsive to these five forces, will gain trust and secure sustained investment for AI initiatives.
Q10. How has KX developed since August 2022?
Ashok Reddy: Since August 2022, KX has been at the forefront of driving accelerated computing for data and AI-driven analytics, catering specifically to AI-first enterprises. Our focus on strategic partnerships has allowed us to deliver cutting-edge solutions tailored to meet the evolving needs of our customers.
Furthermore, our commitment to innovation and customer-centricity has solidified our position as a trusted partner in the AI-driven analytics space. By aligning our efforts with market demands, we continue to lead the way in delivering transformative solutions that drive success for our customers.
……………………………………….
Ashok Reddy, CEO, KX
One of the leading voices in vector databases, search & temporal LLM’s, Ashok KX joined as Chief Executive Officer in August 2022. He has more than 20 years of experience leading teams and driving revenue for Fortune 500 and private equity-backed technology companies. He spent ten years at IBM as Group General Manager where he led the end-to-end delivery of enterprise products and platforms for a diverse portfolio of global customers. In addition, he has held leadership roles at CA Technologies and Broadcom, and worked as a special adviser to digital transformation company Digital.AI where he helped the senior leadership team devise the product and platform vision and strategy.
Resources:
(**) 5 Forces That Will Drive the Adoption of GenAI. Harvard Business Review, Dec 14, 2023.
Related Posts:
On Generative AI. Interview with Maharaj Mukherjee. ODBMS Industry Watch, December 10, 2023.
On Generative AI. Interview with Philippe Kahn. ODBMS Industry Watch, June 19, 2023.
…………………
“Digital ethics must be context specific. To bridge the operationalization gap, we must consider that digital ethics cannot be a one-size fits-all approach.“
Q1. What is your responsibility and your current projects as Director Bioethics & Digital Ethics at Merck KGaA?
Jean Enno Charton: As the Director of Bioethics and Digital Ethics at Merck KGaA, I’m responsible for addressing ethical considerations and questions that arise from rapid advancements in science and technology especially in areas of legal ambiguity and moral complexity. Merck as a leading science and technology company is perpetually at the forefront of pioneering new projects that redefine our offerings and customer interactions. Consequently, my responsibilities and projects are dynamic, evolving in line with Merck’s growth and the ever-changing scientific landscape. Over the recent years I have built up a team as specialists that works as sort of ‘in-house consultancy for ethics.
For example, we have to carefully evaluate which organizations we supply with our life science products, such as genome editing tools, to prevent potential misuse or abuse for unethical purposes. Other recent projects that I am involved in are related to stem cell research, human embryo models, fertility technologies, clinical research, donation programs, and others. My work also includes ensuring data and algorithms at Merck are used in ethically favourable ways that align with our values and principles. Here, we are frequently delving into ethical considerations surrounding AI deployment and data utilization in research and development, data analytics, human resource management and other areas.
In addition, we coordinate with two advisory panels that provide independent external guidance on how to use the scientific and digital innovations developed by Merck KGaA in a responsible manner. These are the Merck Ethics Advisory Panel for Science and Technology (MEAP) and the Digital Ethics Advisory Panel (DEAP). My team and I select the topics and experts for these panels and disseminate their recommendations.
Q2. Recent attempts to develop and apply digital ethics principles to address the challenges of the digital transformation leave organizations with an operationalisation gap. Why?
Jean Enno Charton: The operationalisation gap in digital ethics arises mainly due to the necessarily high level of abstraction of ethical principle frameworks vis-à-vis the granularity needed to answer day-to-day challenges that arise from data and AI use.
Ethical principles are typically formulated at a high level of abstraction. While these principles serve as essential compass points, they lack the specificity needed for practical application. Imagine a map with coordinates indicating the general direction but failing to provide street-level details. Similarly, high level ethical principles guide organizations but fall short when it comes to navigating the intricacies of real-world scenarios.
Moreover, digital ethics must be context specific. To bridge the operationalization gap, we must consider that digital ethics cannot be a one-size fits-all approach. Each organization context – whether it’s healthcare, finance, or human resources – presents unique challenges. In some instances, you may need geopolitical maps and in others you may want geographical maps.
Q3. What are the main challenges in translating high-level ethics frameworks into practical methods and tools that match the organization specific workflows and needs?
Jean Enno Charton: The primary challenge lies in the intellectual capacity required for such translation. In the realm of bioethics and digital health, ethical considerations are intricate and multifaceted, demanding a nuanced approach to each unique scenario.
Organizations must have the capability to create tools and methodologies that are fit for their specific operational needs. Careful planning is essential, entailing a strategic plan of integrating these tools into current systems with minimal interruption and optimal productivity. Furthermore, fostering inter departmental cooperation is crucial to overcome the common challenge of compartmentalization within organizations. Often, the resources necessary for such endeavors are scarce or inadequately allocated.
Additionally, digital ethics presents unique challenges that necessitate a fundamental shift from the traditional model of one-time advisory consultations. The highly automated and dynamic nature of project oversight, coupled with the scale and velocity of data analytics projects, calls for ongoing ethical engagement and innovative approaches to responsibility assignment. In dispersed data analytics teams, attributing ethical accountability is particularly difficult because individual responsibilities can become unclear. Consequently, there is an imperative for developing new methodologies for ethical assessment. These methodologies must be adaptable to the changing landscape and sufficiently nuanced to reflect the complexities of data and AI utilization.
Q3. You helped develop a risk assessment tool called Principle-at-Risk Analysis (PaRA). What is it? And how does it work in practice at Merck KGaA?
Jean Enno Charton: The Principle-at-Risk Analysis (PaRA) is a standardized risk assessment tool developed to bridge high-level ethics frameworks with practical methods and tools that align with specific workflows and needs. At Merck KGaA, PaRA guides and harmonizes the work of the DEAP, ensuring alignment with the company’s Code of Digital Ethics (CoDE).
How does PaRA work?
- Identification: The first step is to identify potential scenarios or decisions in the project seeking guidance from the DEAP where ethical principles could be compromised. These principles include privacy, transparency, fairness, and accountability. At the end of the process, the panel receives a list of potential conflicts between Merck’s CoDE and the project being investigated, enabling a comprehensive review of all relevant ethical concerns.
- Assessment: Once potential risks are identified, they are assessed based on their potential impact on ethical principles. The assessment considers factors such as the severity of the potential harm, the likelihood of occurrence, and the company’s ability to mitigate the risk.
- Mitigation: After assessing the risks, measures are implemented to mitigate or manage them effectively. This may involve adjusting processes, implementing safeguards, or providing additional training and guidance to employees involved in decision-making.
- Monitoring and Review: The PaRA framework emphasizes ongoing monitoring and review of ethical risks to ensure that mitigation measures remain effective. This includes regular audits, feedback mechanisms, and adapting strategies as new risks emerge or circumstances change.
- Integration with Decision-Making: Importantly, the PaRA framework is integrated into the company’s decision-making processes. This ensures that ethical considerations are taken into account when making business decisions, from strategic planning to day-to-day operations.
In practice, we have applied PaRA in various contexts, such as ensuring the comprehensibility of consent forms in data-sharing scenarios at Syntropy, a collaborative technology platform for clinical research. The tool can also be applied across various departments and functions, such as research and development or marketing.
Q4. How can ethics panels make an effective contribution to implementing digital ethics in a commercial organization?
Jean Enno Charton: Ethics panels can play a crucial role in implementing digital ethics in commercial organization by providing expertise, guidance and oversight in navigating complex ethical quandaries associated with digital technologies.
In corporate context of companies like Merck, internal ethics teams may be limited in size, hindering their ability to handle diverse ethical issues effectively. This is where external advisory panels come in by contributing diverse knowledge from technical fields, ethics, sociology, anthropology, and law. This diverse expertise is essential for addressing complex ethical conundrums in areas such as stem cell, patient consent, and AI in HR.
Ethics panels also act impartially in balancing commercial interests and ethical considerations ensuring fair outcomes.
They contribute to developing robust ethical policies and frameworks, drawing from their experience in policy formation, public consultation, or regulatory roles. DEAP, for instance, assisted in developing CoDE, PaRA, among other recent methodological advancements at Merck.
Panels are also well equipped to conduct risk and opportunity assessments to identify potential ethical concerns and prioritize appropriate countermeasures. This approach promotes a more humane technological landscape that aligns opportunities with ethics for a conscientious contribution.
Q5. Merck created a digital ethics panel composed of external experts in digital ethics, law, big data, digital health, data governance and patient advocacy. How do you handle conflict of interests and ensure a neutral approach?
Jean Enno Charton: To address potential conflicts of interests and maintain neutrality, the panel members are required to disclose potential conflicts of interest, affiliations with other organizations, or any other factors that might affect their impartiality. This disclosure is part of an agreement that panel members make with Merck during the recruitment process, ensuring a commitment to good faith actions.
Additionally, Merck employs a transparent selection process for its panel members. Candidates are meticulously chosen through a comprehensive process that prioritizes independence, breadth of expertise that is pertinent to Merck’s diverse portfolio, academic merit, diversity of viewpoints, and the capacity to accurately reflect the views pertinent to specific geographic regions as well as the interests of minority groups that are often underrepresented in ethical assessments.
Furthermore, our Code of Ethics and PaRA standardizes the panel’s activities, providing structured support in the decision-making processes, thereby ensuring the utmost neutrality and integrity in the panel’s operations.
Q6. How do you avoid the risk that an ethics panel will form an isolated entity within the company?
Jean Enno Charton: Avoiding the risk of an ethics panel becoming an isolated entity within a company involves a delicate balance of independence and integration. It is crucial for such a panel to maintain a certain level of detachment to offer unbiased independent opinions. Yet, it’s equally important to prevent the panel from becoming isolated. At Merck, we achieve this balance by fostering a sense of inclusion within both the Merck and scientific communities.
Our panel members, while not Merck employees, often have a long-standing relationship with the company. We ensure they are well informed about Merck’s activities and specific use cases, enabling them to offer valuable insights without internal biases.
To promote transparency, we communicate the panel’s activities, decisions, and recommendations within the company through minutes or other appropriate communication methods. This open communication helps other stakeholders understand and value the panel’s role, mitigating the risk of isolation. It also nurtures a sense of belonging and contribution to the broader scientific endeavor at Merck.
Q7. Can you share some details of a best practice you have developed using the PaRA tool?
Jean Enno Charton: One of the best practices we have developed using PaRA tool involves integrating it with other ethical frameworks, such as CoDE, and collaborating with experts including DEAP and my own team to ensure a comprehensive approach to ethical considerations.
It’s important to note that the PaRA tool doesn’t encompass all ethical aspects. Therefore, revisiting and reassessing the tool’s output is sometimes necessary to identify any overlooked or missed elements. This practice has significantly enhanced our effective utilization.
Q8. What are the main lessons you have learned?
Jean Enno Charton: One of the main lessons I have learned is the importance of scientific rigor in developing and implementing a risk analysis tool like PaRA. While a principled framework provides a foundation, it must be supported by rigorous scientific and ethical analysis to ensure its effectiveness and meaningfulness. This involves conducting thorough research, gathering relevant data, and engaging with experts in various fields to inform the development of the tool.
Another key lesson is the necessity of garnering support from senior leadership for the success of the tool. Without the backing of senior leadership, it would have been challenging – if not impossible – to develop and implement PaRA effectively.
Further, I have learned the need for flexibility and sensitivity to the specific needs of individual departments within the organization. While overarching ethical principles guide the development of the tool, it’s essential to recognize that different departments may face unique challenges and priorities. As such, the tool must be adaptable enough to accommodate these varying needs while upholding ethical standards consistently across the organization. This flexibility ensures that the tool remains relevant and applicable across diverse departments and scenarios.
Q9. How do you make sure that the recommendations developed on the basis of the Principle-at-Risk Analysis are really enforced and not ignored in practice?
Jean Enno Charton: From a governance perspective Our role is advisory rather than decision or enforcement. We aim to convince our collaborators to adopt the recommendations from the panels and implement the principles laid out in the Merck’s Code of Digital Ethics.
Departments and individuals bear the primary responsibility for adhering to these principles. However, we support department leaders in creating mechanisms for effective enforcement.
For instance, we have created a handbook-style self-assessment tool for project managers working with generative AI to identify and mitigate ethical risks during project development. Additionally, we have embedded semi-automated ethics assessment processes into existing project management structures within data analytics.
We also actively engage with department leaders to encourage follow-up and implementation of recommendations to ensure accountability and transparency. When recommendations are not feasible due to various constraints, we communicate the constraints to the ethics panel, valuing their input while outlining implementation barriers.
Q10. What are the challenges and limitations of such an approach?
Jean Enno Charton: A significant challenge of this approach is the voluntary nature of compliance. Unlike traditional compliance departments that enforce rules and regulations, we rely on voluntary adherence to ethical principles. While the voluntary approach elevates ethics to a higher moral standard, it also means that there’s no direct mechanism for enforcement. This voluntary aspect can pose challenges in ensuring consistent adherence across all departments and levels of the organization. This has also to do with the culture and code of conduct we live at Merck – a still largely family-owned company with a long-term, generationally-thinking approach on doing responsible business.
Qx Anything else you wish to add?
Jean Enno Charton: (WHAT’S THE BIG TAKEAWAY)
…………………………………..
Dr. Jean Enno Charton, Director Bioethics & Digital Ethics , Merck KGaA
Dr. Jean Enno Charton is Director Digital Ethics & Bioethics at Merck. After a brief stint in the biotech industry, he has been with Merck since 2014 – first in the Research & Development department of the Healthcare division (Medical Affairs), later as Chief of Staff of the Chief Medical Officer Healthcare. Since 2019, he has built up the independent Digital Ethics & Bioethics department and is responsible for the topic across all divisions.
Jean Enno Charton studied biochemistry at the University of Tübingen and obtained his doctorate in cancer research at the University of Lausanne; his research experience includes stays at the Canadian Science Center for Human and Animal Health and Harvard Medical School.
Related Posts
On Generative AI. Interview with Philippe Kahn. ODBMS Industry Watch, June 19, 2023
………………………………………..
“Open source is reshaping the technological landscape, and this holds particularly true for AI applications. As we progress into AI, we will witness the proliferation of open-source systems, from large language models to advanced AI algorithms and improved database systems.“
Q1. What is your definition of a Vector Database?
Charles Xie: A vector database is a cutting-edge data infrastructure designed to manage unstructured data. When we refer to unstructured data, we specifically mean content like images, videos, and natural language. Using deep learning algorithms, this data can be transformed into a novel form that encapsulates its semantic representation. These representations, commonly known as vector embeddings or vectors, signify the semantic essence of the data. Once these vector embeddings are generated, we store them within a vector database, empowering us to perform semantic queries on the data. This capability is potent because, unlike traditional keyword-based searches, it allows us to delve into the semantics of unstructured data, such as images, videos, and textual content, offering a more nuanced and contextually rich search experience.
Q2. Currently, there are a multitude of vector databases on the market. Why do they come in so many versions?
Charles Xie: When examining vector database systems, disparities emerge. Some, like Chroma, adopt an embedded system approach akin to SQLite, offering simplicity but lacking essential functionalities like scalability. Conversely, systems like PG Vector and Pinecone pursue a scale-up approach, excelling in single-node instances but limiting scalability.
As a seasoned database engineer with over two decades of experience, I stress the complexity inherent in database systems. A systematic approach is vital when assessing these systems, encompassing components like storage layers, storage formats, data orchestration layers, query optimizers, and execution engines. Considering the rise of heterogeneous architectures, the latter must be adaptable across diverse hardware, from modern CPUs to GPUs.
From its inception, Milvus has embraced heterogeneous computing, efficiently running on various modern processors like Intel and AMD CPUs, ARM CPUs, and Nvidia GPUs. The integration extends to supporting vector processing AI processes. The challenge lies in tailoring algorithms and execution engines to each processor’s characteristics, ensuring optimal performance. Scalability, inevitable as data grows, is a crucial consideration addressed by Milvus, supporting both scale-up and scale-out scenarios.
As the vector database gains prominence, its appeal to vendors stems from its potential to reshape data management. Therefore, transitioning to a vector database necessitates evaluating its criticality to business functions and anticipating data volume growth. Milvus stands out for both scenarios, offering consistent, optimal performance for mission-critical services and remarkable cost-effectiveness as data scales.
Q3. In your opinion when does it make sense to transition to a pure vector database? And when not?
Charles Xie: Now, let’s delve into the considerations for transitioning to a pure vector database. It’s crucial to clarify that a pure vector database isn’t merely a traditional database with a vector plugin; it’s a purposefully designed solution for handling vector embeddings.
There are two key factors to weigh. Firstly, assess whether vector computing and similarity search are critical to your business. For instance, if you’re constructing a RAG solution integral to millions of users daily and forming the core of your business, the performance of vector computing becomes paramount. In such a situation, opting for a pure vector database system is advisable. It ensures consistent, optimal performance that aligns with your SLA requirements, especially for mission-critical services where performance is non-negotiable. Choosing a vector database system guarantees a robust foundation, shielding you from unforeseen surprises in your regular database services.
The second crucial consideration is the inevitable increase in data volume over time. As your service runs for an extended period, the likelihood of accumulating larger datasets grows. With the continuous expansion of data, cost optimization becomes an inevitable concern. Most pure vector database systems on the market, including Milvus, deliver superior performance while requiring fewer resources, making them highly cost-effective.
As your data volume escalates, optimizing costs becomes a priority. It’s common to observe that the bills for vector database services grow substantially with the expanding dataset. In this context, Milvus stands out, showcasing over 100 times more cost-effectiveness than alternatives such as PG Vector, OpenSearch, and other non-native web database solutions. The cost-effectiveness of Milvus becomes increasingly advantageous as your data scales, making it a strategic choice for sustainable and efficient operations.
Q4. What is the initial feedback from users of Vector Databases?
Charles Xie: Reflecting on our beginnings six years ago, we focused primarily on catering to enterprise users. At the time, we engaged with numerous users involved in recommendation systems, e-commerce, and image recognition. Collaborations with traditional AI companies working on natural language processing, especially when dealing with substantial datasets, provided valuable insights.
The predominant feedback we received emphasized the enterprise sector’s specific needs. These users, being enterprises, possessed extensive datasets and a cadre of proficient developers. They emphasized deploying a highly available and performant vector database system in a production environment, a requirement often seen in large enterprises where AI was gaining traction.
It’s important to note that independent AI developers were not as prevalent during that period. AI, being predominantly in the hands of hyper-scalers and large enterprises, meant that the cost of developing AI algorithms and applications was considerably high. Around six years ago, hyper-scalers and large enterprises were the primary users of vector database systems, given their capacity to afford dedicated teams of AI developers and engineers. This context laid the foundation for our initial focus and direction.
In the last two years, we’ve witnessed a remarkable shift in the landscape of AI, marked by the breakthrough of modern AI, particularly the prominence of large language models. Notably, there has been a significant surge in independent AI developers, with the majority comprising teams of fewer than five individuals. This starkly contrasts the scenario six years ago when the AI development scene was dominated by large enterprises capable of assembling teams of tens of engineers, often including a cadre of computer science PhDs, to drive AI application development.
The transformation is striking—what was once the exclusive realm of well-funded enterprises can now be undertaken by small teams or even individual developers. This democratization of AI applications marks a fundamental shift in accessibility and opportunities within the AI space.
Q5. Will semantic search be performed in the future by ChatGPT instead of using vectors and a K-nearest neighbor search?
Charles Xie: Indeed, the foundation models we encounter, such as Chat GPT and vector databases, share a common theoretical underpinning—the embedding vector abstraction. Both Chat GPT and vector database systems leverage embedding vectors to encapsulate the semantic essence of the underlying unstructured data. This shared data abstraction allows them to make sense of the information and perform queries effectively. Across large language models, AI models, and vector database systems, a profound connection exists rooted in the utilization of the same data abstraction—embedding vectors.
This connection extends further as they employ identical metrics, primarily relying on distance metrics like Euclidean or cosine distance. Whether within Chat GPT or other large language models, using consistent metrics facilitates the measurement of similarities among vector embeddings.
Theoretically, a profound connection exists between large language models like Chat GPT and various vector databases, stemming from their shared use of embedding vector abstraction. The workload division between them becomes apparent—they both excel at performing semantic and k-nearest neighbor searches. However, the noteworthy distinction lies in the cost efficiency of these operations.
While large language models and vector databases tackle the same tasks, the cost disparity is significant. Executing semantic search and k-nearest neighbor search in a vector database system proves to be approximately 100 times more cost-effective than carrying out these operations within a large language model. This substantial cost difference prompts many leading AI companies, including OpenAI, to advocate for using vector databases in AI applications for semantic search and k-nearest neighbor search due to their superior cost-effectiveness.
Q6. There seems to be a need from enterprises to have a unified data management system that can support different workloads and different applications. Is this doable in practice? If not, is there a risk of fragmentations of various database offerings?
Charles Xie: No, I don’t think so. To illustrate my point, let’s consider the automobile industry. Can you envision a world where a single vehicle serves as an SUV, sedan, truck, and school bus all at once? This has yet to happen in the last 100 years of the automobile industry, and if anything, the industry will be even more diversified in the next 100 years.
It all started with the Model T; from this, we witnessed the birth of a great variety of automobiles commercialized for different purposes. On the road, we see lots of differences between SUVs, trucks, sports cars, and sedans, to name a few. A closer look at all these automobiles reveals that they are specialized and designed for specific situations.
For instance, SUVs and sedans are designed for family use, but their chassis and suspension systems are entirely different. SUVs typically have a higher chassis and a more advanced suspension system, allowing them to navigate obstacles more easily. On the other hand, sedans, designed for urban areas and high-speed driving on highways, have a lower chassis for a more comfortable driving experience. Each design serves a specific goal.
Looking at all these database systems, we see that many design goals contradict each other. It’s challenging, if not impossible, to optimize a design to meet all these diverse requirements. Therefore, the future of database systems lies in developing more purpose-built and specialized ones.
This trend is already evident in the past 20 years. Initially, we had traditional relational database systems. Still, over time, we witnessed the emergence of big data solutions, the rise of NoSQL databases, the development of time series database systems, graph database systems, document database systems, and now, the ascent of vector database systems.
On the other hand, certain vendors might have an opportunity to provide a unified interface or SDK to access various underlying database systems—from vector databases to traditional relational database systems. There could be a possibility of having a unified interface.
At Milvus, we are actively working on this concept. In the next stage, we aim to develop an SQL-like interface tailored for vector similarity search in vector databases. We aim to incorporate vector database functionality under the same interface as traditional SQL, providing a unified experience.
Q7. What does the future hold for Vector databases?
Charles Xie: Indeed, we are poised to witness an expansion in the functionalities offered by vector database systems. In the past few years, these systems primarily focused on providing a single functionality: approximate nearest neighbor search (ANN search). However, the landscape is evolving, and in the next two years, we will see a broader array of functionalities.
Traditionally, vector databases supported similarity-based search. Now, they are extending their capabilities to include exact search or matching. You can analyze your data through two lenses: a similarity search for a broader understanding and an exact search for detailed insights. By combining these two approaches, users can fine-tune the balance between obtaining a high-level overview and delving into specific details.
Obtaining a sketch of the data might be sufficient for certain situations, and a semantic-based search works well. On the other hand, in situations where minute differences matter, users can zoom in on the data and scrutinize each entry for subtle features.
Vector databases will likely support additional vector computing workloads, such as vector clustering and classification. These functionalities are particularly relevant in applications like fraud detection and anomaly detection, where unsupervised learning techniques can be applied to cluster or classify vector embeddings, identifying common patterns.
Q8. And how do you believe the market for open source Vector databases will evolve?
Charles Xie: Open source is reshaping the technological landscape, and this holds particularly true for AI applications. As we progress into AI, we will witness the proliferation of open-source systems, from large language models to advanced AI algorithms and improved database systems. The significance of open source extends beyond mere technological innovation; it exerts a profound impact on our world’s social and economic fabric. In the era of modern AI, with the dominance of large language models, open-source models and open-source vector databases are positioned to emerge victorious, shaping the future of technology and its societal implications.
Q9. In conclusion, are Vector databases transforming the general landscape, not just AI?
Charles Xie: Indeed, vector databases represent a revolutionary technology poised to redefine how humanity perceives and processes data. They are the key to unlocking the vast troves of unstructured data that constitute over 80% of the world’s data. The promise of vector database technology lies in its ability to unleash the hidden value within unstructured data, paving the way for transformative advancements in our understanding and utilization of information.
………………………………………………..
Charles Xie is the founder and CEO of Zilliz, focusing on building next-generation databases and search technologies for AI and LLMs applications. At Zilliz, he also invented Milvus, the world’s most popular open-source vector database for production-ready AI. He is currently a board member of LF AI & Data Foundation and served as the board’s chairperson in 2020 and 2021. Charles previously worked at Oracle as a founding engineer of the Oracle 12c cloud database project. Charles holds a master’s degree in computer science from the University of Wisconsin-Madison.
Related Posts
On Zilliz Cloud, a Fully Managed AI-Native Vector Database. Q&A with James Luan. ODBMS.org,JUNE 15, 2023
On Vector Databases and Gen AI. Q&A with Frank Liu. ODBMS.org, DECEMBER 8, 2023
“ Five years from now, today’s AI systems will look archaic to us. In the same way that computers of the 60s look archaic to us today. What will happen with AI is that it will scale and therefore become simpler, and more intuitive. And if you think about it, scaling AI is the best way to make it more democratic, more accessible.“
Q1. What are the innovations that most surprised you in 2023?
Raj Verma: Generative AI is definitely the talk of the town right now. 2023 marked its breakthrough, and I think the hype around it is well founded. Few people knew what generative AI was before 2023. Now everyone’s talking about it and using it. So I was quite impressed by the takeup of this new technology.
But if we go deeper, we have to acknowledge that the rise of AI would not have been possible without significant advancements in how large amounts of data are stored and handled. Data is the core of AI and what is used to train LLMs. Without data, AI is useless. To have powerful generative AI that gives you answers, predictions and content right at the moment you need it, you need real-time data, or data that is fresh, in motion and delivered in a matter of milliseconds. The interpretation and categorization of data are therefore crucial in powering LLMs and AI systems.
In that sense, you will notice a lot of hype around Specialized Vector Databases (SVDB), which are independent systems that you plug into your data architecture designed to store, index and retrieve vectors, or multidimensional data points. These are popular because LLMs are increasingly relying on vector data. Think of vectors as an image or a text converted into a stored data point. When you prompt an AI system, it will look for similarities in those stored data points, or vectors, to give you an answer. So vectors are really important for AI systems and businesses often believe that a database focused on just storing and processing vector data is essential for AI systems.
However, you don’t really need SVDBs to power your AI applications. In fact, loads of companies have come to regret their use because, as an independent system, they result in redundant data, excessive data movement, increasing labor and licensing costs and limited query power.
The solution is to store all your data — structured data, semi-structured data based on JSON, time-series, full-text, spatial, key-value and vector data — in one database. And within this system have a powerful vector database functionality that you can leverage to conduct vector similarity search.
All this to say that, I’ve been impressed at the speed in which we are developing ways to power generative AI. We’re experimenting based on its needs and quickly figuring out what works and doesn’t work.
Q2. What is real-time data and why is it essential for AI?
Raj Verma: Real time is about what we experience in the now. It is access to the information you need, at the moment you need it, delivered together with the exact context you need to make the best decision. To experience this now, you need real-time data — data that is fresh and in motion. And with AI, the need for real-time data — fast, updated and accurate data — is becoming more apparent. Because without data, AI is useless. And when AI models are trained on outdated or stale data, you get things like AI bias or hallucinations. So, in order to have AI that is powerful, and that can really help us make better choices, we need real time data.
With the use of generative AI expanding beyond the tech industry, the need for real-time data is more urgent than ever. This is why it is important to have databases that can handle storage, access and contextualization of information. At SingleStore, our vision is that databases should support both transactional (OLTP) and analytical (OLAP) workloads, so that you can transact without moving data and put it in the right context — all of which can be delivered in millisecond response times.
Q3. One of the biggest concerns around AI is bias, the idea that existing prejudices in the data used to train AI might creep into its decisions, content and predictions. What can we do to mitigate this risk?
Raj Verma: I believe humans should always be involved in the training process. With AI, we must be both student and teacher, allowing it to learn from us, and in that way continuously give it input so that it can give us the insight we need. There are many laudable efforts to develop Hybrid Human AI models, which basically incorporate human insight with machine learning. Examples of hybrid AI include systems in which humans monitor AI processes through auditing or verification. Hybrid models can help businesses in several ways. For example, while AI can analyze consumer data and preferences, humans can jump in to guide how it uses that insight to create relevant and engaging content.
As developers, we must also be very cognizant of where the data used to train LLMs comes from. And in this sense, being transparent about where it comes from helps, because the systems can be held accountable and challenged if biased data does creep into the training process. The important thing here is also to know that an AI system is only as good as the data that is trained on.
Q4. The popularity and accessibility of generative artificial intelligence (gen AI) has made it feel like the future we see in science fiction movies is finally at our doorstep. And those science fiction movies have sowed much worry about AI being dangerous. Is this Science fiction vision of AI becoming true?
Raj Verma: Don’t expect machines to take over the world, at least not any time soon. AI can process and analyze large amounts of data and generate content based on that, at a much faster pace than we humans can. But they are still very dependent on human input. The idea that human-like robots will come to rule the world makes for great fiction movies, but it’s far from becoming a reality.
That doesn’t mean that AI isn’t dangerous — and we have a responsibility to discern discerning which threats are real.
AI poses an unprecedented risk in fueling the spread of disinformation because it has the capacity to create authentic looking content. Distinguishing between content generated by AI and that created by humans will become increasingly challenging. AI can also pose cybersecurity threats. You can trick ChatGPT into writing malicious code, or use other generative AI systems to enhance ransomware. And AI can worsen current malicious trends that have surfaced with social media. I personally worry that AI systems will exploit the attention economy and spur higher levels of social media addiction. This can have terrible consequences on teenagers’ mental health. As a father of two, I am deeply concerned about this.
These are the threats that we should worry about. And we humans are capable of mitigating these risks. We should always be involved in AI’s development, audit it and pay special attention to the data that we use to train it.
Q5. You are quoted saying that ” without data, AI wouldn’t exist—but with bad or incorrect data, it can be dangerous.” How dangerous can AI be?
Raj Verma: Generative AI is like a superhuman who reads an entire library of thousands of books to answer your question, all in a matter of seconds. If it doesn’t have access to that library, and if that library doesn’t have the latest books, magazines and newspapers, then it cannot give you the most relevant information you need to make the best decision possible. This is a very simple explanation of why, without data, AI is useless. Now imagine that library is full of outdated books that were written by white supremacists during the civil war. The information you are going to get from this AI system is going to guide your decisions, and you are going to make some very bad decisions. You are going to make biased decisions, and you’re going to perpetuate biases that already exist in society. That’s how AI can be dangerous, and that is why we need AI systems to have access to the most updated, accurate data out there.
Q6. Should AI be Regulated? And if yes, what kind of regulation?
Raj Verma: The issue is, it’s hard to regulate something that is still developing. We just don’t know what AI will look like, in its entirety, in the future. So we want to avoid regulation hampering the development of this technology. That doesn’t mean that there aren’t standards that can be applied globally. Data regulation is key, since data is the backbone of AI. Data regulation can be based on the principle of transparency, which is key to generate trust in AI and our ability to hold this technology and its developers accountable should something go wrong. To achieve transparency you need to know where the data in the AI system is coming from. So, proper documentation of the data used to train LLMs is something we can regulate. You also must be able to explain the reasoning behind an AI system’s solutions or decisions. These must be understandable by humans. And there’s also transparency in how you present the AI system to users. Do users know that they are talking to an AI robot and not a human? We can regulate data transparency without imposing excessive measures that could hamper AI’s development.
Q7. There is no global approach on AI regulation. Several Countries in the world are in various stages of evolving their approach to regulating AI. What are the practical consequences of this?
Raj Verma: A global scale regulation of AI is incredibly challenging. Each country’s social values will be reflected in the way they approach regulating this new technology. The EU has a very strong approach to consumer protection and privacy, which is probably why it authored the first significant widespread attempt to regulate AI in the world. I don’t believe we will see such a wide sweeping legislation in the US, a country that values innovation and market dynamics. The US, we will see a decentralized approach to regulation, with maybe some specific decrees that seek to regulate its use in specific industries, like healthcare or finance.
Many worry that the EUs new AI act will become another poster child of the Brussels effect, where firms end up adopting the EU’s regulation, in absence of any other, because it saves costs. Yet the Brussels effect might not exactly happen with the AI act, particularly because firms might want to use different algorithms in the first place. For example, marketing companies will want to use different algorithms for different geographic areas because consumers behave differently depending on where they live. It won’t be hard then for firms to have their different algorithms comply with different rules in different regions.
All this to say that we should expect different AI regimes around the world. Companies should prepare for that. AI trade friction with Europe is likely to emerge, and private companies will advance their own “responsible AI” initiatives as they face a fragmented global AI regulatory landscape.
Q8. How can we improve the way we gather data to feed LLMs?
Raj Verma: We need to make sure LLMs are up to date. Open source LLMs that are trained on large, publicly available data are prone to hallucinate because at least part of their data is outdated and probably biased. There are ways to fix this problem, including Retrieval Augmented Generation (RAG), which is a technique that uses a program to retrieve contextual information from outside the model, immediately feeding it to the AI system. Think of it as an open book test where the AI model, with the help of a program (the book), can look up information specific to the question it is being asked about. This is a very cost effective way of updating LLMs because you don’t need to retrain it all the time and can use it in case-specific prompts.
RAG is central to how we at SingleStore are bringing LLMs to date. To curate data in real time, it needs to be stored as vectors, which SingleStore allows users to do. That way you can join all kinds of data and deliver the specific data you need in a matter of milliseconds.
Q9. What is the evolutionary path you think AI will go through? When we look back 5-10 years from now, how will we look at genAI systems like ChatGPT?
Raj Verma: Five years from now, today’s AI systems will look archaic to us. In the same way that computers of the 60s look archaic to us today. What will happen with AI is that it will scale and therefore become simpler, and more intuitive. And if you think about it, scaling AI is the best way to make it more democratic, more accessible. That is the challenge we have in front of us, scaling AI, so that it works seamlessly in giving us the exact insight we need to improve our choices. I believe this scaling process should revolve around information, context and choice, what I call the trinity of intelligence. These are the three tenets that differentiate AI from previous groundbreaking technologies. They are also what help us experience the now in a way that we are empowered to make the best choices. Because this is our vision at SingleStore, we focus on developing a multi-generational platform which you can use to transact and reason with data in millisecond response times. We believe this is the way to make AI more powerful because with more precise databases that can deliver information in real time, we can power the AI systems that will really help us make the best choices as humans.
………………………………………..
Raj Verma is the CEO of SingleStore.
He brings more than 25 years of global experience in enterprise software and operating at scale. Raj was instrumental in the growth of TIBCO software to over $1 billion in revenue, serving as CMO, EVP Global Sales, and COO. He was also formerly COO at Apttus Software and Hortonworks. Raj earned his bachelor’s degree in Computer Science from BMS College of Engineering in Bangalore, India.
Related Posts
Achieving Scale Through Simplicity + the Future of AI. Raj Verma. October 31st, 2023
On Generative AI. Interview with Maharaj Mukherjee. ODBMS Industry Watch, December 10, 2023
On Generative AI and Databases. Interview with Adam Prout. ODBMS Industry Watch, October 9, 2023
__________________
