awesome-copilot/skills/fabric-lakehouse/references/getdata.md

Data Factory Integration

Microsoft Fabric includes Data Factory for ETL/ELT orchestration:

• 180+ connectors for data sources
• Copy activity for data movement
• Dataflow Gen2 for transformations
• Notebook activity for Spark processing
• Scheduling and triggers

Pipeline Activities

Activity	Description
Copy Data	Move data between sources and Lakehouse
Notebook	Execute Spark notebooks
Dataflow	Run Dataflow Gen2 transformations
Stored Procedure	Execute SQL procedures
ForEach	Loop over items
If Condition	Conditional branching
Get Metadata	Retrieve file/folder metadata
Lakehouse Maintenance	Optimize and vacuum Delta tables

Orchestration Patterns

Pipeline: Daily_ETL_Pipeline
├── Get Metadata (check for new files)
├── ForEach (process each file)
│   ├── Copy Data (bronze layer)
│   └── Notebook (silver transformation)
├── Notebook (gold aggregation)
└── Lakehouse Maintenance (optimize tables)

---