Tobiko

Ever wonder how top data teams validate complex transformations at scale? 🧐Fivetran Data Architect Ezra Y. Lee breaks down how his team does it with #SQLMesh's table_diff feature in his new guest blog for Tobiko. Ezra's team has fully integrated SQLMesh's table_diff into their workflows, and in his post (linked below), he walks through how it's improved their testing processes, development speed, and data consistency. He covers everything from day-to-day use cases---like verifying schema changes and fixing duplicate records---to powering automated table comparisons across environments and making it a core part of Fivetran's QA and model development process. 👉 To see how SQLMesh table_diff has helped the Fivetran analytics architecture team, complete with practical tips, scripts, and real-world applications, 𝐫𝐞𝐚𝐝 𝐭𝐡𝐞 𝐟𝐮𝐥𝐥 𝐩𝐨𝐬𝐭: https://lnkd.in/ehXh2iAZ

𝗗𝗲𝘃𝗲𝗹𝗼𝗽𝗲𝗿 𝗲𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲 𝗺𝗮𝘁𝘁𝗲𝗿𝘀. 🌿 As a company, we’re rooted in our open-source community and guided by a philosophy of building products that make data workflows feel like second nature. That’s what we set out to do with our latest SQLMesh changes (which, by default, are also included in our hosted, enterprise-ready Tobiko Cloud). We overhauled the CLI output—from the language it uses to how it’s visually structured—to make building and debugging pipelines faster, more intuitive, and easier on the eyes 👀 This update gives you: ✅ Instant clarity on what SQLMesh is doing: a centralized view to see if your plan ran tests, built models, or triggered backfills as you intended ✅ Clear breakdowns of operations by stage (physical layer → execution → virtual layer) ✅ Smart defaults and optional verbosity, so you can move on when runs are successful, but have the color to deep-dive if they break ✅ Inline audit results (with ❌ or ✅) to help you avoid guesswork around blocking and non-blocking issues ✅ Thoughtfully reworded terms that better reflect what’s going on in your work or debugging flow—so you can minimize time spent asking questions or consulting docs We want your DevEx to be useful, beautiful, and trustworthy, to keep you in flow, and to make you feel that 'my CLI didn’t just tell me what happened, it also gave me the information I need to be confident in taking the next step.’ 👉 Read our blogpost for more details, and let us know what you think in the Tobiko community Slack—both linked in the comments below. #DataEngineering #SQLMesh #OpenSource #DevOps #TobikoCloud

Behind every great conference is an even greater party 🥃 Because the most valuable ideas and connections happen when the slides stop and the sliders start 🍔 Come find the Tobiko team on both Day 1 𝗮𝗻𝗱 Day 2 of Data Council - we'll be co-hosting a Data Renegades Happy Hour on 4/22 with Recce, Datacoves and bauplan, and joining Rill Data on 4/23 for a technical roundtable with ClickHouse, StarTree and MotherDuck. 👉 Register for the Data Renegades Happy Hour at https://lnkd.in/eVx4YTkC 👉 Register for Rill Data's roundtable & dinner at https://lnkd.in/dMPmREQQ #DataCouncil #DataEngineering #DataTransformation #SQLMesh #TobikoCloud

🏎️ Tobiko is headed to Data Council! Our co-founder and CTO Tobias (Toby) Mao will be joining Rill Data CEO Michael Driscoll for a roundtable discussion on post-modern ELT and real-time databases along with Jordan Tigani of MotherDuck, Yury Izrailevsky of ClickHouse, and Kishore Gopalakrishna of StarTree 🗣️ No conference ticket required. ✍️ Just register at the link in comments below to join us for food, networking, and an honest, technical deep dive with some of the leading minds and companies in next-gen data solutions. We hope to see you there! #DataTransformation #DataEngineering #TobikoCloud #SQLMesh #DataAnalytics #DataWarehouses #ELT

👉 𝐒𝐚𝐯𝐞 𝐭𝐡𝐞 𝐝𝐚𝐭𝐞 for the very first episode of Tobiko's monthly live demo! We're hosting our inaugural webinar 𝐭𝐨𝐦𝐨𝐫𝐫𝐨𝐰, 𝐀𝐩𝐫𝐢𝐥 9, at 10am PST. You'll get an in-depth tour of our newly launched Tobiko Cloud solution and hear from Morgan Valentine and our sales architect Sung Won Chung on why it changes the data transformation game with: ✨ Intuitive native scheduling that leverages our unique state-aware architecture ✨ Best-in-class data security and governance ✨ Advanced debugging and impact analysis tools ✨ Other features designed for enterprise users of any scale And to our open-source community members, we're also bringing the latest and greatest updates we've been making in SQLMesh. So whether you're new to SQLMesh, or curious about its hosted version with new enterprise capabilities, 𝐫𝐞𝐠𝐢𝐬𝐭𝐞𝐫 𝐧𝐨𝐰 at https://lnkd.in/gGUVzyxS to join us! ⏰ #DataEngineering #SQLMesh #DataTransformation #TobikoCloud

🚨 Tobiko’s biggest news of the year (for now): 𝗦𝗤𝗟𝗠𝗲𝘀𝗵’𝘀 𝙚𝙣𝙩𝙚𝙧𝙥𝙧𝙞𝙨𝙚 𝘂𝗽𝗴𝗿𝗮𝗱𝗲 It’s been a journey to arrive at this point—but after two years of constant innovation (and internalizing our community’s feedback), 𝘄𝗲 𝗵𝗮𝘃𝗲 𝗹𝗮𝘂𝗻𝗰𝗵𝗲𝗱 𝗧𝗼𝗯𝗶𝗸𝗼 𝗖𝗹𝗼𝘂𝗱! 💫 Our hosted, enterprise-ready transformation product suite amplifies all the capabilities data engineers love in SQLMesh—with added features to help data teams easily collaborate and scale their workflows at companies of any size: 🧐 𝗚𝗿𝗮𝗻𝘂𝗹𝗮𝗿, 𝗔𝗰𝘁𝗶𝗼𝗻𝗮𝗯𝗹𝗲 𝗢𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗶𝗹𝗶𝘁𝘆 » 𝘊𝘶𝘴𝘵𝘰𝘮 𝘢𝘭𝘦𝘳𝘵𝘴: Configure notifications to trigger on failed runs and custom behaviors via Slack, PagerDuty, or email. Deep links let you trace any issues directly in the Tobiko Cloud UI » 𝘜𝘯𝘪𝘧𝘪𝘦𝘥 𝘥𝘦𝘣𝘶𝘨𝘨𝘦𝘳: Our new debugger ‘view’ consolidates error logs, recent activity, code definitions, and more into a single view—giving you all the context color to resolve issues quickly » 𝘊𝘰𝘴𝘵 𝘵𝘳𝘢𝘤𝘬𝘦𝘳: We make warehouse expenses easier to monitor and manage with a tracker that isolates and keeps tabs on compute in BigQuery, Snowflake, and other engines, and pinpoints your most costly models 📅 𝗕𝘂𝗶𝗹𝘁-𝗶𝗻 𝗢𝗿𝗰𝗵𝗲𝘀𝘁𝗿𝗮𝘁𝗶𝗼𝗻—𝗡𝗼 𝗖𝗜/𝗖𝗗 𝗥𝗲𝗾𝘂𝗶𝗿𝗲𝗱 » 𝘕𝘢𝘵𝘪𝘷𝘦 𝘴𝘤𝘩𝘦𝘥𝘶𝘭𝘦𝘳: Automatically schedule and group jobs without the hassle of manual setup. We also introduced a concurrency feature to prevent long-running jobs from bottlenecking other parallel workflows » 𝘗𝘢𝘶𝘴𝘦 𝘮𝘰𝘥𝘦𝘭 𝘳𝘶𝘯𝘴: Temporarily halt model or task runs for maintenance, troubleshooting, or agile iterations—without disrupting any other part of your pipeline 🔐 𝗦𝗦𝗢 𝗮𝗻𝗱 𝗘𝗻𝘁𝗲𝗿𝗽𝗿𝗶𝘀𝗲-𝗚𝗿𝗮𝗱𝗲 𝗦𝗲𝗰𝘂𝗿𝗶𝘁𝘆 𝗮𝗻𝗱 𝗙𝗹𝗲𝘅𝗶𝗯𝗶𝗹𝗶𝘁𝘆 » 𝘐𝘴𝘰𝘭𝘢𝘵𝘦𝘥 𝘗𝘺𝘵𝘩𝘰𝘯 𝘦𝘯𝘷𝘪𝘳𝘰𝘯𝘮𝘦𝘯𝘵𝘴: Keep your runs secure—and clean—by preventing dependency conflicts between jobs » 𝘏𝘺𝘣𝘳𝘪𝘥 𝘥𝘦𝘱𝘭𝘰𝘺𝘮𝘦𝘯𝘵: Choose where your data lives. Running Tobiko Cloud in hybrid mode preserves your data and operations within your own infrastructure, and is ideal for enterprises with strict compliance protocols 🔺 𝗔𝗱𝘃𝗮𝗻𝗰𝗲𝗱 𝗜𝗺𝗽𝗮𝗰𝘁 𝗗𝗲𝘁𝗲𝗰𝘁𝗶𝗼𝗻 » 𝘚𝘮𝘢𝘳𝘵 𝘤𝘩𝘢𝘯𝘨𝘦 𝘤𝘭𝘢𝘴𝘴𝘪𝘧𝘪𝘤𝘢𝘵𝘪𝘰𝘯: Make fewer unnecessary backfills. We enriched SQLMesh’s column-level lineage and breaking change detection to automatically determine if modified columns are used downstream » 𝘊𝘳𝘰𝘴𝘴-𝘥𝘢𝘵𝘢𝘣𝘢𝘴𝘦 𝘥𝘪𝘧𝘧𝘪𝘯𝘨: Our new hashing algorithm enables data teams migrating between vendors to compare datasets across warehouses without the cost of full joins 💡 𝗧𝗿𝘆 𝗧𝗼𝗯𝗶𝗸𝗼 𝗖𝗹𝗼𝘂𝗱 𝘁𝗼𝗱𝗮𝘆 If your team wants to boost productivity, strengthen data governance, and use a transformation tool that’s tailored for your workflows (rather than the other way around), 𝗹𝗲𝘁’𝘀 𝘁𝗮𝗹𝗸 → https://lnkd.in/eD6-AAkD #DataEngineering #SQLMesh #TobikoCloud #DataTransformation

Last week, our team at Tobiko held our inaugual GROUP BY conference—our first annual gathering dedicated to the data engineering community—and we’re still buzzing ⚡ from the conversations, learnings, and opportunity to meet with many of our customers, partners, and advocates, some for the first time in person. 💥 We’re energized by the growing value companies are assigning to their data, and by the new generation of tools emerging around an ever larger and more intricate data ecosystem—tools that are more nimble, collaborative, and focused on making data transformation, observability, modeling, and security fit the way data teams work, rather than the other way around. We believe it’s the most exciting the data space has ever been, and we’re thrilled to be a part of the evolution 🧬 reshaping the way we build and scale data systems . 🙌 A massive thank you to customers, peers, and friends from our VC and open-source communities who joined us as speakers Justin Kolpak Joseph Lane Reuven Gonzales Janelle Teng Madison Faulkner Wei Lien Dang Petr Janda Alon Nafta Christophe O. Your insights and support were instrumental in making this event a success. Stay tuned for more exciting launches and announcements soon to come! 📣 #dataengineering #datatransformation #SQLMesh #TobikoData #Avios #Databricks #KaribaLabs #BessemerVenturePartners #NEA #SYNQ #Foundational

✨ In case you need (yet) another reason to come to Tobiko's data engineering conference, we've added an exciting lineup of speakers from Bessemer Venture Partners, New Enterprise Associates (NEA) and Unusual Ventures to our VC panel! 🏃🏻♂️ Register now at https://lu.ma/j15cttbw for your spot at 𝗚𝗥𝗢𝗨𝗣 𝗕𝗬 𝗧𝗼𝗯𝗶𝗸𝗼 𝗻𝗲𝘅𝘁 𝗧𝗵𝘂𝗿𝘀𝗱𝗮𝘆, 𝗠𝗮𝗿𝗰𝗵 𝟮𝟬 in San Francisco. See you there!

Tobiko's first data conference is happening in less than 3 weeks! 🏃 Alon Nafta, CEO of Foundational, is the latest to join our list of guest speakers. Register at https://lu.ma/j15cttbw to see how we're working with Foundational to automate data contracts across the tech stack - and with other partners, like Databricks, Avios, Kariba Labs, SYNQ, to improve velocity and productivity in data engineering!

SQLMesh + ClickHouse: Transform your data at ClickHouse speed 🚀 We raced here to share the news that SQLMesh (by Tobiko) now supports ClickHouse to bring precise, error-free data transformation to one of the fastest analytical databases today 🏎️ With SQLMesh, data teams can safely develop, test, and deploy changes to their models—with little to no reprocessing or disruption to production. Pair this with ClickHouse’s lightning-fast query performance, and you get real-time analytics at unparalleled scalability and efficiency ⚡ Want to see it in action? Read our integration announcement post (link in comments) to learn more about how SQLMesh and ClickHouse work together. #SQLMesh #ClickHouse #TobikoData #DataTransformation #Analytics #DataEngineering