This is the third blog in a series looking at the need for DW Automation focusing on Synapse SQL Pools (dedicated).
- In the first blog we looked at why we need DW Automation tools and what they look like:
https://prodata.ie/2021/09/14/the-need-for-dw-automation-1-3/ - In the second blog we looked at a maturity model for DW automation with six levels of Automation we see in the field.
https://prodata.ie/2021/10/03/a-dw-automation-maturity-model-2-3/
DW Automation in Action Now – Demo Time
For the Data Platform Virtual Summit I did a 1 hour end to end demo of DW Automation techniques covering:
- Source Code and getting Started
- Extract Automation (piping)
- Logging, Lineage, and Error Handling (Fat, aka Enterprise features)
- Transform and Load Automation (Business Logic and more complex piping)
- Moving from fixed workflow to Dynamic Orchestration.
This is covered in the video below.
Note: you will need to register for a free accout on data platform virtual summit to see this (and lots of other content!).
Let me know what you think of the concepts. Happy to chat on twitter (@bob_duffy) where you will see me discussing data architecture with lots of other MVPs, BI and data professionals in the data community online.
Want to see more ?
I’m doing a similar demo (more focused on extract and load) at #DataWeekender on Sat 6th November at 1PM Irish Time
https://www.dataweekender.com/schedule
Shameless plug
Our consultants do of course commercially build EDW and Enterprise BI projects at scale for large enterprises based on SQL, especially on ERP platforms like SAP and M3. Happy to discuss commercial projects too 😉
One thought on “DW Automation demo for Synapse SQL Pools (3/3)”