mrpaulandrew / procfwk

A cross tenant metadata driven processing framework for Azure Data Factory and Azure Synapse Analytics achieved by coupling orchestration pipelines with a SQL database and a set of Azure Functions.

https://mrpaulandrew.com/category/azure/data-factory/adf-procfwk/

mrpaulandrew/procfwk Issues

Extend Monitoring and Troubleshooting
Updated 8 months ago1
End of support Azure Functions 3.x
Updated a year ago
Could not find object: linkedService.SupportDatabase
Closed 2 years ago1
Synapse Orchestrator: InvalidTokenAuthenticationAudience
Updated 2 years ago1
Dynamic Json parameter in framework pipelines parameters (ex. Tumbling Window Trigger Parameters)
Updated 2 years ago
Restarting a Batch
Updated 2 years ago2
Dependency Within a Stage
Updated 2 years ago4
Reference issue in "CheckForBlockedPipelines.sql" file!
Closed 2 years ago2
ADF Pipeline Parameter value seems to be cached. When I run the pipeline its automatically taking the Previous run values even if I enable=1 new PipelineID and make Enabled=0 for the previous PipelineID
Closed 3 years ago4
Retry "Check For Alerts" activity
Updated 3 years ago
Add Failure Handling Activities for All Database Calls Dealing with Timeouts and Connection Loss
Updated 3 years ago
[procfwk].[CheckForBlockedPipelines] has an unresolved reference to object [procfwk].[CurrentExecution].[AdfPipelineRunId]
Closed 3 years ago4
Corrupt file DeployProcFwkComponents.ps1?
Updated 4 years ago
Passing Local Execution ID value to a worker pipeline
Updated 4 years ago
Option to run Pipelines in sequence in 03-Child
Updated 4 years ago
Pipeline status running after timeout setting pipeline result
Updated 4 years ago1
Question: Is there a way to migrate pre-existing ADF to this framework?
Closed 4 years ago2
Enhancements for batch mode
Updated 4 years ago
Import / Export Framework Metadata
Updated 4 years ago1
Harden Synapse Pipeline Interactions
Updated 4 years ago
Finish first pass GitHub Pages Documentation
Updated 4 years ago2
Replace Management API Calls in Check Running Utils Pipeline with Function Call
Updated 4 years ago
Switch Email Alerting Call to Use Utils Pipeline
Updated 4 years ago1
Retried ExecutionId and Attempt Number
Updated 4 years ago
Add Parameter Set Support to be Used with Execution Batches
Updated 4 years ago
Automatic cancellation of worker pipelines when an child or infant pipeline is cancelled
Updated 4 years ago3
Extend Execution Stages with a new Level - Execution Batches
Closed 4 years ago2
Add Support for Framework Controlled Pipeline Cancellations of Workers
Updated 4 years ago1
Deployment Files of ADF - missing pipelines
Closed 4 years ago
Could not find object: linkedService.SupportDatabase on DeployProcFwkComponents.ps1
Closed 4 years ago18
Separate Metadata Integrity and Clean Up
Closed 4 years ago
Capture the Last Pipeline Parameter Value Used in the Parameters Table
Closed 4 years ago
Check for Running Pipeline in ADF without Database Support
Closed 4 years ago1
Create Generic Throw Exception Pipeline
Closed 4 years ago
Validate Worker Pipeline Name as part of Infant Run
Closed 4 years ago
Triggering the Parent pipeline multiple times could overwrite preceding in progress runs
Closed 4 years ago4
Allow multiple CurrentExecution to be managed by framework?
Closed 4 years ago10
Extend Pipeline Table to Include LogicalUsageValue Attribute for Reused Worker Pipelines
Updated 4 years ago
Use Secure Activity Inputs/Outputs
Closed 4 years ago
Update Activity Timeout Values
Closed 4 years ago1
Use PowerShell Checker Script to Expose Cosmetic Bits Missed in Framework Factory
Closed 4 years ago
Avoid pointless execution run after a CleanPreviousRun results in all pipelines having the status Success
Closed 4 years ago3
Add Worker Pipeline Auto Retry Options
Updated 4 years ago
Add NUnit Tests for Framework Restart Scenarios
Updated 4 years ago
Add NUnit Tests for Framework Clean Up Process
Updated 4 years ago
Refactor Common Expressions into Pipeline Variables
Closed 4 years ago2
Refactor Child & Infant Pipelines to Make Worker Execute Simpler and Isolated
Closed 4 years ago
Add Unit Tests for Cancelled Worker Pipelines
Updated 4 years ago
Default parameter value incorrect for Parent pipeline. Update execution log activity.
Closed 4 years ago
Overhaul Power BI Execution Run Dashboard
Updated 4 years ago