Wie können KI und ML Ihren Datenanalyse-Workflow verbessern?

in my experience this is true however, without business context or the people that have built the source system AI will only connect the more obvious data points.

I often hear the following, “before we start our AI journey we need to get our data right.” The pursuit of perfect data is noble, but becomes a stumbling block for progress. However, without reliable data, there is no AI. Years ago we invested hundreds of hours training an ML outlier detection model that is used to strip data that can skew the analytics engine for a logistics company. That same model could have been built in dozens of hours with today’s AI.

I agree with Tim. To state the obvious quality of data going in is core to the success of AI despite the cleaning and wrangling. A thoughtful clean process to input data becomes very essential to have the output and benefits of AI as a whole

If we have a domain-specific language model, it can help us identify and transform the data according to the business use case. This idea was always there, even before the popularization of LLMs. Domain-specific topic modelling and document classifications were a few ways to automate date preparation for business use cases.

In my experience, we analyze data to help us improve efficiencies and optimize resources. Correct data analysis enables one to focus on the future with a clear understanding.

Even though there are a lot of AI enable tools to help with the analytics and exploration part of the data pipeline, this is a judgmental activity, There are no templates for how the data should be perceived, to start with. Even human cognition shows confusion when confronted with unrelated terms (I have published this using survey and pair embeddings). To make it clear, not finding a pattern is not the same as the pattern never existed, just that the pattern is not there in the data sample. It also depends on what level we need to explore, E.g. looking for say causality, we surely need to leverage on complex networks, centrality metrics for each data attribute etc, even then the expected outcome might not be forthcoming.

This is true in theory but I haven't seen it being deployed successfully in mid-tier business yet. I think it's coming but without some kind of semantic layer it's hard to explore.

This is too vague to be useful. What is unsupervised vs supervised and what does reinforcement learning do. what are hyperparameters - I think this part is way too high level to be useful

Hi Tim, perhaps some basic definitions will be helpful to people that are new to AI or could use a quick refresher: Supervised learning uses labeled training data, and unsupervised learning does not. More simply, supervised learning models have a baseline understanding of what the correct output values should be. Reinforcement learning is an ML technique that trains software to make decisions to achieve the most optimal results. It mimics the trial-and-error learning process that humans use to achieve their goals. Hyperparameters are ML training variables. You set them through multiple experiments, where you pick a set of hyperparameters and run them through your model. This is called hyperparameter tuning.

I‘d add that the future of the internet will mostly likely be generative interfaces tailored to the individual and powered by AI that retrieves the knowledge from an API. Doesn‘t have to be language based. You can also generate an interface pretty easy.

AI and ML can enhance data security by improving encryption, biometric authentication, threat detection, providing real-time monitoring, adaptive measures, and efficient incident response, albeit with limitations. They cannot fully replace human judgment in assessing unforeseen situations and may exhibit biases, necessitating careful implementation. Human oversight is crucial to monitor for algorithmic biases and a comprehensive strategy involving regular updates, model explainability and a multi-layered security approach is essential to address these challenges.

Although AI and ML can generate new data, it doesn’t mean it will be accurate or useful. It all depends on the training data used to develop the AI and ML models. If the input variables and data are biased in any way, the new data will be as well. For example, facial recognition software do worse at processing images from black people compared to white people. Audio transcription tools do a terrible job if the speakers have different accents. They even confuse male and female voices. Expressions of emotions are culturally determined. To recognize them correctly requires understanding of the culture and specific contexts in which they are being expressed. AI and ML models reduce emotions to a handful of inaccurate biased stereotypes.

This is kinda vague and throws a lot of different things together. What is true is that for LLMs there seems to be no downside of using synthetic data - see Microsoft’s paper „Textbooks are enough“. The question remains if this is true for example for image generation AI. There seems to be some „model collapse“ but overall not a huge problem. Most importantly sometimes the data is not even needed like suggested in the description above. There are some papers indicating that large language models like GPT4 outperform narrow AIs in more complex tasks currently (Minecraft Agents for example). I would also argue that many companies don‘t have so valuable data and benefit more if a sophisticated LLM retrieves their knowledge via a Vector DB.

AI and ML can significantly enhance your data analytics workflow by automating data processing, reducing the time spent on repetitive tasks, and increasing accuracy. They enable predictive analytics, helping to forecast trends and identify patterns that might be missed by traditional methods. Machine learning algorithms can also refine models over time, improving insights as more data is processed. Additionally, AI-driven tools can generate more detailed visualizations, making it easier to communicate findings effectively to stakeholders.