Integrating Sharepoint Online with OneLake in Fabric

Its common to have spreadsheets in SharePoint and want to ingest them into OneLake in Fabric. Typically this could be master data, mappings, budgets or any type of data that does not reside within a core enterprise system. We’ve developed a Python class that can be used in Fabric Notebooks that wraps the Graph API…

How to create a service principal to automate Fabric

Service Principals are essential to using the Fabric REST API for automation. In this blog well show you how to create a service principle that can be used to automate API calls to PowerBI such as refresh. Once you have created the service principle the next article shows you how to use the service principle to…

Fabric DW Automation Now Available!

We launched today our Fabric DWA Framework Accelerator. This is the result of over 3,500 hours of work migrating our Synapse DWA Framework to Fabric and also re-designing the template library system to make it extensible enough to handle any and all Fabric artefacts like Pipelines, Dataflows, Notebooks as well as traditional SQL Procedures and…

Sequential Identity in Fabric

Why do we need sequential identity in data warehousing? Currently Fabric doesnt support Identity columns. Even if/when it does, I assume that the numbers won’t be sequential as most MPPs such as SqlPools generate different ranges per “node” as per Synapse, resulting in numbers that may be unique but look somewhat random. https://azurede.com/2021/07/31/comparing-sql-server-and-synapse-analytics-identity-columns/ First up,…

Success with Azure Synapse Analytics: Mastering Data Warehousing

In today’s data-driven world, organizations must effectively manage and utilize vast amounts of information to gain insights and stay competitive. To achieve this, companies are increasingly turning to data warehouses, a centralized repository that stores and organizes structured and unstructured data from various sources.

How to Refresh Fabric Datasets using Python API

If you are like me and wants to automate the task to refresh Fabric datasets to report latest data to users then this article is for you. The automation can be done using Python REST API and the article will show the steps to follow. The full python script is available to use at GitHub.…

Connecting to Fabric using SqlClient

Microsoft Fabric doesnt yet support SqlLogins, but you can still connect programatically by using the Azure.Identity Classes which support both Managed Identity (if run from somewhere supporting that), or Visual Studio authentication if in debug or unit test mode. The examples below are c#, but similar works in Python too. Below is a code sample…

Real move on Data Lake using Rest API on ADF

There are many times where I’ve needed to move a file using Azure’s Data Factory and found it weird that the only native option provided is to copy the file, then delete it. Many reading know that disk level renames are actually almost instantaneous (just a rewrite the the files metadata / partition tree). A…

Bug – Synapse SqlPool Language and dates

We found this interesting bug on Dedicated SQL Pool when working with British style language. If you use Dynamic SQL to change the language, then selects on the date fail with a conversion error, but prints work. Weird… An example repro is below If you use a hard coded language statement as opposed to dynamic,…

SqlDW / Synapse – TABLE Variable Compatibility

Azure Synapse SQL does not support the the use of TABLE variable. Therefore a workaround is to use Temporary Tables in SqlDW. If we run the below statement in Synapse SQL then it willa throw syntax error as below: Alternative solution to achive the same goal as Table Variable in SqlDW / Synapse SQL is…

Enabling Managed Identity Authentication on Azure Functions in Data Factory

This blog shows you how to enable Managed Identity on an Azure Function call from ADF, this can help secure the Function by adding a second authentication method on top of the API Key. Enabling these options enforces https on all function calls, ADF handles authentication and so no custom headers, tokens or keys are…

Loading…

Something went wrong. Please refresh the page and/or try again.

always on Azure Cloud consolidation Cube DW DWA EDW ERP Events Fabric Featured Finance Financial Reporting Irish Economic Crisis Licencing Load Test Maestro Many to Many MasterClass MCM mdx Modelling MOLAP OLAP Partitioning Performance Personal Power BI PowerPivot scom SQL SQLBits sqldw SQL IO SQL Saturday SQL Server SSAS SSIS storage Syanpse Synapse Tabular TSQL Virtualization

Subscribe to Prodata Blog

Get new content delivered directly to your inbox.