mirror of
https://github.com/github/awesome-copilot.git
synced 2026-02-20 02:15:12 +00:00
36 lines
1.0 KiB
Markdown
36 lines
1.0 KiB
Markdown
### Data Factory Integration
|
|
|
|
Microsoft Fabric includes Data Factory for ETL/ELT orchestration:
|
|
|
|
- **180+ connectors** for data sources
|
|
- **Copy activity** for data movement
|
|
- **Dataflow Gen2** for transformations
|
|
- **Notebook activity** for Spark processing
|
|
- **Scheduling** and triggers
|
|
|
|
### Pipeline Activities
|
|
|
|
| Activity | Description |
|
|
|----------|-------------|
|
|
| Copy Data | Move data between sources and Lakehouse |
|
|
| Notebook | Execute Spark notebooks |
|
|
| Dataflow | Run Dataflow Gen2 transformations |
|
|
| Stored Procedure | Execute SQL procedures |
|
|
| ForEach | Loop over items |
|
|
| If Condition | Conditional branching |
|
|
| Get Metadata | Retrieve file/folder metadata |
|
|
| Lakehouse Maintenance | Optimize and vacuum Delta tables |
|
|
|
|
### Orchestration Patterns
|
|
|
|
```
|
|
Pipeline: Daily_ETL_Pipeline
|
|
├── Get Metadata (check for new files)
|
|
├── ForEach (process each file)
|
|
│ ├── Copy Data (bronze layer)
|
|
│ └── Notebook (silver transformation)
|
|
├── Notebook (gold aggregation)
|
|
└── Lakehouse Maintenance (optimize tables)
|
|
```
|
|
|
|
--- |