
Sequential Identity in Fabric
Why do we need sequential identity in data warehousing? Currently Fabric doesnt support Identity columns. Even if/when it does, I assume that the numbers won’t be sequential as most MPPs such as SqlPools generate different ranges per “node” as per Synapse, resulting in numbers that may be unique but look somewhat random. https://azurede.com/2021/07/31/comparing-sql-server-and-synapse-analytics-identity-columns/ First up,…

Success with Azure Synapse Analytics: Mastering Data Warehousing
In today’s data-driven world, organizations must effectively manage and utilize vast amounts of information to gain insights and stay competitive. To achieve this, companies are increasingly turning to data warehouses, a centralized repository that stores and organizes structured and unstructured data from various sources.

Fabric – Refresh PBI Dataset using Python API
To invoke Power BI Dataset refresh using Python, use ‘requests’ library to make HTTP requests to the API. Prerequisites: Client ID, Client Secret and Tenant ID are needed to generate access token Functions to get PowerBI Workspace ID and Dataset ID Function to Invoke PowerBI dataset refresh using workspace ID and dataset ID Function to…

Connecting to Fabric using SqlClient
Microsoft Fabric doesnt yet support SqlLogins, but you can still connect programatically by using the Azure.Identity Classes which support both Managed Identity (if run from somewhere supporting that), or Visual Studio authentication if in debug or unit test mode. The examples below are c#, but similar works in Python too. Below is a code sample…
Real move on Data Lake using Rest API on ADF
There are many times where I’ve needed to move a file using Azure’s Data Factory and found it weird that the only native option provided is to copy the file, then delete it. Many reading know that disk level renames are actually almost instantaneous (just a rewrite the the files metadata / partition tree). A…
Recreating a Power BI GIT Repos in Azure DevOps
In a previous blog, I showed how to shrink a GIT repos using git commands and the opensource bfg-cleaner to remove history of large PBIX files.https://prodata.ie/2021/10/23/shrinking-azure-repos-bloat-due-to-powerbi/ Sometimes the patient is just to sick and its too messy to clean-up, so we nuke from orbit. Essentially creating a fresh repos and getting developers to re-clone. Sure…
Bug – Synapse SqlPool Language and dates
We found this interesting bug on Dedicated SQL Pool when working with British style language. If you use Dynamic SQL to change the language, then selects on the date fail with a conversion error, but prints work. Weird… An example repro is below If you use a hard coded language statement as opposed to dynamic,…

SqlDW / Synapse – TABLE Variable Compatibility
Azure Synapse SQL does not support the the use of TABLE variable. Therefore a workaround is to use Temporary Tables in SqlDW. If we run the below statement in Synapse SQL then it willa throw syntax error as below: Alternative solution to achive the same goal as Table Variable in SqlDW / Synapse SQL is…
Enabling Managed Identity Authentication on Azure Functions in Data Factory
This blog shows you how to enable Managed Identity on an Azure Function call from ADF, this can help secure the Function by adding a second authentication method on top of the API Key. Enabling these options enforces https on all function calls, ADF handles authentication and so no custom headers, tokens or keys are…

Import Json using SQL Dedicated Pool in Azure Synapse Analytics (SQLDW)
This blog shows how to easily Extract Json and XML objects and Arrays into SQDW using COPY INTO SQLDW is a bit behind other vendors like snowflake when it comes to being able to query Json natively off a data lake as they have native support. To get around this we simply load the Json…

Bob Duffy – User Group and Event Sessions for 2022
If you are running a SQL/Data User group or event I’m always happy to do a 50-60 minute virtual talk. Let me know bob@prodata.ie or @bob_duffy on twitter. 2022 Session Options 01. SqlPool Maintenance Fundamentals SQL Pools now support auto create status. But they do not support auto update stats and don’t quite have the…

SQL Storage Fundamentals for Azure VMs
Thx to anyone looking at my session at #SsqlBits on SQL Storage fundamentals for Azure VMs. Here is a copy of my slide deck if you want to see any of the links or content.
Loading…
Something went wrong. Please refresh the page and/or try again.
always on Azure backups Cloud consolidation Cube DW EDW ERP Events Featured Finance Financial Reporting Irish Economic Crisis Licencing Load Test Maestro Many to Many MasterClass MCM mdx Modelling MOLAP OLAP Partitioning Performance Personal Power BI PowerPivot Power View scom SQL SQLBits sqldw SQL IO SQL Saturday SQL Server SSAS SSIS storage Syanpse Synapse Tabular TSQL Virtualization
Subscribe to Prodata Blog
Get new content delivered directly to your inbox.