Skip to main content
200 questions
Newest Active Bountied Unanswered
Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

Can you send a message from Azure Synapse Notebook to Service Bus topic, with local auth disabled in service bus? Is Managed Identity available to do that? Or can connection string also be used in a ...
user32079711's user avatar
  • 1
0 votes
0 answers
61 views

I am attempting to use the Python SDK to create a linked service to an Azure Blob Storage account in my Synapse Workspace. I have some code which creates a dictionary whose structure matches exactly ...
j_marvin's user avatar
  • 47
0 votes
0 answers
55 views

I'm trying to retrieve a SAS token for a storage account from code running in an Azure Synapse workspace. Normally, one would obtain this via a linked service. However, in my case, the storage account ...
DxG's user avatar
  • 231
0 votes
0 answers
72 views

My company demands that I connect from Synapse Spark (pyspark notebooks) to Azure Event Hubs using managed identity only (preferably the system assigned managed identity). I have this kind of code in ...
Alex Serban's user avatar
  • 73
0 votes
0 answers
37 views

I'm trying to read from Azure kusto source using below code in Azure synapse notebook. df = spark.read \ .format("com.microsoft.kusto.spark.synapse.datasource") \ .option("kustoCluster&...
DxG's user avatar
  • 231
0 votes
1 answer
67 views

I got my data in PowerBI by connecting to Azure Synapse, where I need to transform a column in PowerBI where the column (text) contains list of numeric values eg. [0,1.0,0.05] in a single row needs to ...
Megha M's user avatar
  • 1
0 votes
2 answers
138 views

I have a Spark Pool template working, and I would like to select/reference an existing Spark Configuration that was created in a given Synapse Workspace as below: Select in here: But if I run the ...
Guilherme Matheus's user avatar
  • 649
0 votes
0 answers
33 views

I am not able to connect to external REST API with azure synapse. Sample code is as below from the notebook which is throwing a Failed to establish a new connection: [Errno 101] Network is unreachable ...
Enem's user avatar
  • 31
0 votes
1 answer
173 views

Requirement: Grant read-only (or specific) access to all tables in an Azure Synapse Dedicated SQL Pool for a set of users by assigning them to a group in Azure Entra ID. Execution: For example, say I'...
Vivek's user avatar
  • 5
0 votes
1 answer
101 views

I have a delta table in a directory in a storage account and I am creating an external table in azure synapse using this query IF NOT EXISTS (SELECT * FROM sys.external_file_formats WHERE name = '...
Asfandyar Abbasi's user avatar
  • 105
0 votes
1 answer
114 views

I am writing output to a Azure Synapse table where the table contains a varbinary(8000) column. When writing using spark it gives error that UNSUPORTED_DATATYPE as I am trying to limit length from ...
Nitish Sharma's user avatar
  • 11
0 votes
1 answer
168 views

I have a CSV file and it looks like below: (Column Delimiter : +| and row delimiter : &|\n) This file has two problems : 1st - Both row and column delimiter is 2 characters, and this does not ...
sam's user avatar
  • 1,304
0 votes
0 answers
135 views

I am trying to copy data from Synapse workspace to Azure SQL db using ADF. I am using User assigned managed identity for authentication. Linked service connection is getting succeed, but while ...
Hemant Kumar's user avatar
  • 75
1 vote
1 answer
108 views

In synapse serverless SQL pool OR dedicated SQL pool, can row level security(RLS) be implemented based on table column value and logged in identity (azure AD group or AD user) say, I have table rows ...
Sri's user avatar
  • 79
0 votes
1 answer
284 views

I'm trying to create an external table on Synapse serverless pool to access a delta table on ADLS gen 2. I first created an external source as shown here. --Create a database scoped credential using ...
Anupam Chand's user avatar
  • 2,779
0 votes
0 answers
107 views

I had a requirement of implementing a Selective deployment of azure synapse analytics pipelines from one env.to other (DEV to QA) using azure devops CI/CD, git.I have tried to implement it using arm ...
sreekar hari's user avatar
  • 1
0 votes
1 answer
178 views

I'm trying to transform a key/value data into column using Azure synapse Data Flow. Basically this: "scoreIndeces": { "66": 0.2, "67": 0.3, "68": 0.4, "70&...
Balaji Dev's user avatar
  • 3
0 votes
1 answer
200 views

I am migrating stored procedure from SQL Dedicated to Synapse Serverless SQL Pool. The procedure queries a table and stores the results into a temporary table. Then it iterates over the temporary ...
Renato's user avatar
  • 1
0 votes
1 answer
113 views

I am trying to add a table in Azure Synapse using EF Core 9.0, but I am getting an error: Parse error at line 2, column 1: Incorrect syntax near EXEC. ActionItemStatus Class: public class ...
Sonal Khatri's user avatar
  • 1
0 votes
1 answer
172 views

We have 3 synapse workspaces (dev, test, maint) connects to it's own branch in the same Azure DevOPs repository. We also have feature branches based off the dev branch. All our development is done ...
theprof86's user avatar
  • 33
0 votes
1 answer
203 views

I am having issues with a series of pipelines that build our data platform Spark databases hosted in Azure Synapse. The pipelines host dataflows which have 'recreate table' enabled. The dataflows ...
NITHIN B's user avatar
  • 334
0 votes
1 answer
139 views

I am new to Azure Workspace and I am trying to start a basic spark app. from pyspark.sql import SparkSession from pyspark.sql.functions import col, expr # Initialize Spark session spark = ...
Jed's user avatar
  • 1,051
0 votes
3 answers
514 views

I'm new the Azure Lakehouse and warehouse. I'm trying to grasp the concept in Microsoft Fabric workspace so that I can choose either lakehouse or warehouse to be my destination. I understand that ...
variable's user avatar
  • 9,896
0 votes
1 answer
196 views

REST API call from synapse notebook to any other services outside azure not working ( power bi , Google) ex: test_url = "https://www.google.com" response = requests.get(test_url) print("...
Madhu Tallapaneni's user avatar
  • 19
-1 votes
1 answer
104 views

I have a problem which i'm currently working on and i can't wrap my head around it after reading the official documentation and seeing topics online, i'll try ot be as clear as possible: I'm using ...
Porchetta17's user avatar
  • 11
-1 votes
1 answer
159 views

I have a blob storage container where i have folder and subfolder like structure example: Container: XYZ Folder: yyyy-MM-dd subFoller: HH There is one folder created for every day and in that hourly ...
TA01's user avatar
  • 15
0 votes
1 answer
536 views

In current on Prem etl we stage incremental data (based on date modified, held in a control table), and UPSERT, that is - update existing records in the final table and insert new records into the ...
variable's user avatar
  • 9,896
-1 votes
1 answer
100 views

I have data in azure data lake gen2 parquet files. Using Azure synapse analytics I have created SQL pool. Then, via linked service I'm connecting to parquet file. Then, I have created multiple views ...
variable's user avatar
  • 9,896
0 votes
0 answers
72 views

I have an azure function which need to talk to Azure Synapse instance, all of them are not allowed to talk to internet. I am trying to use Managed Identity (with DefaultAzureCredential) in Azure Func ...
Sampath's user avatar
  • 45
0 votes
1 answer
309 views

Is there any SQL query that could help me get the size of each schemas present in our Azure Data Warehouse service and also to find when was it last accessed? I checked and found one query below which ...
SRP's user avatar
  • 1,223
1 vote
1 answer
289 views

I have a PL_Fetch pipeline with copy data activity for fetching my sql scripts in my azure synapse analytics. first of all, I made a REST API linked service with the Azure Synapse Workspace I am ...
Joan's user avatar
  • 25
0 votes
1 answer
489 views

I am working on Azure DevOps and Azure Synapse Analytics. Right now, I have the issue to integration Azure DevOps with Azure Synpase Analytics together. How could I use the yaml file and azure ...
Joan's user avatar
  • 25
0 votes
1 answer
196 views

Is it possible to load a .yaml file into a Azure Synapse Sparkr Notebook? How do I load a .yaml file holding credentials from ADLS Gen2 directory?
CFCJB John's user avatar
  • 33
0 votes
1 answer
267 views

How do I set this locationid value dynamically in the relative URL of my DF Rest Source: My design pattern is: I need to pass the locationid from a query lookup to the ForEach. This should loop ...
CFCJB John's user avatar
  • 33
0 votes
1 answer
74 views

I encountered an issue while using Azure Synapse's Lookup Activity to call a stored procedure that deletes records from a table. Despite the activity completing almost instantaneously, no records were ...
user15382501's user avatar
  • 31
1 vote
1 answer
363 views

I have written parquet files in a path like this on the blob storage container: https://<my_blob_storage_accnt>.dfs.core.windows.net/current-snapshot/businesspartner=P1, https://<...
user1330974's user avatar
  • 2,626
0 votes
1 answer
82 views

Using synapse data flow I want to transform the json format bellow into a tabular format with one column as date and the other as Total { "2023-12-11T00:00:00+01:00": 1272, "2023-12-...
Euclides Jaime's user avatar
  • 3
0 votes
1 answer
69 views

My requirement is I have 3000 records in my DB for each 'custID' I wanted to call webcatity and the response I parse and update in the table by using lookup city there is also a case where I am ...
user15382501's user avatar
  • 31
0 votes
1 answer
74 views

AzureSynpase I am calling webActity in for loop, as I am required to call webActity for each record in a table which had 'custid', Now I see some 'custid' is not present in server DB it returns a 400 ...
user15382501's user avatar
  • 31
1 vote
1 answer
281 views

How can we query Azure Synapse Lake database within Azure SQL database? Is there a way to add Synapse lake database as an external data source in Azure SQL database I have few tables present in Azure ...
Praj's user avatar
  • 11
1 vote
1 answer
393 views

created Linked service to Azure SQL database in Synapse Analytics that uses Authentication type " System Assigned Managed Identity". Than referencing it in PySpark Notebook. Method ...
Balbeer Singh's user avatar
  • 11
0 votes
1 answer
112 views

I am trying to filter files containing str or containing stn. if i type -str in the wildcard file path it returns me files containing str and if type -stn, it returns me files containing stn how can I ...
Mishal's user avatar
  • 13
0 votes
1 answer
615 views

I'm working on a project in Synapse and need to assign roles to users such that certain people can see specific data while others cannot. Here's what I've tried so far: Creating a role using the ...
Moody_girl's user avatar
  • 85
0 votes
1 answer
146 views

Following Scala code in Azure Synapse should print: Hello, World!. But instead, it prints: defined object Geeks. Question: What could be the issue and how can we fix it? object Geeks { // Main ...
nam's user avatar
  • 24.3k
1 vote
1 answer
266 views

I'm new to Azure Synapse Analytics and working with CI/CD pipelines. Currently, I'm exploring how to parameterize pipeline parameters. My current code does not have a template-parameters-definition....
SiuN's user avatar
  • 13
0 votes
1 answer
188 views

In my Sink I'm running a Pre SQL Script to delete data after a certain date but I'm having trouble using parameters in the script. Here is my date parameter variable inside dataflow and below is my ...
noobatexcel's user avatar
  • 41
1 vote
1 answer
282 views

I am trying to access the sql script node in my pipeline activities using python. The idea is to export sql-script to this node in ASA (Azure Synapse Analytics) using python. The connection with ...
Ahmed Basiouny's user avatar
  • 27
0 votes
1 answer
673 views

I have a task where I need to connect to external URL from Azure synapse and get few data. We were doing that using synapse web activity and using basic authentication. Now the provider has asked us ...
Shanky's user avatar
  • 634
0 votes
1 answer
188 views

I am trying to drop a view from my synpase Notebook using a connection to a Serverless SQL Pool. I followed the instructions from this post: Access our built-in serverless SQL pool from our notebook ...
Moody_girl's user avatar
  • 85
1 vote
1 answer
860 views

We are currently trying to access our built-in serverless SQL pool from our notebook. Our goal is to be able to drop views from the notebook, which is something we need to put into our pipeline as per ...
Moody_girl's user avatar
  • 85

1
2 3 4