In a previous blog I talked about the need for DW Automation in order to achieve that holy grail of Agility, Extensibility and perhaps Robustness with our data architecture. This blog is now focusing on some different levels of maturity. … Continue reading A DW Automation Maturity Model (2/3)
Many folks build star schema data warehouses and the supporting ecosystem of Semantic Models, Business Intelligence and/or ML Analytics. This is a quick note on how to assess how you are with automation, regardless of what tools you are using: … Continue reading The need for DW Automation (1/3)
TLDR The short answer is YES – if you have >= 60 million rows, then a poorly maintained column store was 100% slower due to trim fragmentation and 64% slower due to the delta row store with some updates in … Continue reading Do we need to maintain Columnstores in SQL Pools ?
One recent realization for myself working with Synapse Dedicated SQL Pools is that while they have auto create stats, there is no auto update stats. So statistics will become stale faster than your bread left in the toaster. Rather than … Continue reading Finding Outdated and Missing Stats in Synapse SQL Pools
I wrote this in 2010, before Clustered Column Store Indexes, SQL Pools (so rule 3 looks a bit odd), and before date data type was mainstream, but the rest of these still look quite Thou Shalt use something that exists … Continue reading Top 10 SQL Partition Commandments
Synapse SQL Dedidcated Pools (aka SQLDW) does not support comments in views or procs in the same was as standalone SQL. This is annoying as comments can be very useful with tracking changes and lineage of objects, especially when used … Continue reading Synpase SQL Tip 2 – Comments in Views/Procs
When loading a Star Schema Datawarehouse it is very common to need to insert rows into the a dimension based on exceptions. For example: Inferred Members (aka early arriving facts) Unknown Members (missing data) A Typical Query may look something … Continue reading Synapse TSQL Tip 1 – Use Double Defensive Inserts
While a lot of projects in Azure may be using SQLDB or SQLDW (aka Dedicated Pools) we do still have a lot of customers running SQL on a VM (aka IaaS). One common task is setting up data disks. Typically … Continue reading Adding Disks to Azure VM with Powershell
In CI/CD Azure Synapse Analytics – part 1 we have covered: Setting up git source control in Synapse Studio The difference in main collaboration and workspace publish branch In this blog we are going to cover: How to create build … Continue reading CI/CD For Azure Synapse Analytics – Part 2
Do you also wonder how to do continuous integration (CI) and continuous deployment (CD) for Synapse Analytics? But first, lets talk about basic, what is CI/CD is simple terms. CI/CD is one of the best practices of agile methodology and … Continue reading CI/CD for Azure Synapse Analytics – Part 1
Now that MSFT have released Synapse as an Integrated framework of tools for Data Engineering we have two different ways to provision our enterprise data warehouse in a dedicated sql pool. Which is the best one to use ? Firstly … Continue reading Standalone or Integrated flavour of your Dedicated Pool (sqldw)
Something went wrong. Please refresh the page and/or try again.