Target Introduction Infrastracture Integration Runtimes Private Link and Managed Network Programmatically Create Data Factory Redundacy Git Management For Connectors LinkedService / Connector / Dataset / Integration Runtime Dataflow / Pipe…

Introduction OneLake OneLake integration Azure Databricks Integrate OneLake with Azure Databricks Ingest data into OneLake and analyze with Azure Databricks Introduction昨年の 11月より一般提供が開始された Microsoft Fabric について、個人検…

Introduction What dbt Core ? Let's Try Preset Install dbt Core and dbt init dbt connection databricks Create models and run dbt core integration Databricks job IntroductionDatabricks と dbt core を使用して、簡易的なデータモデリングを行う内…

Introduction Agenda What is platform engineering ? Platform Implementation Patterns Platform best practices / guidance Next Action IntroductionAWS re:Invent のセッションの内容をレポート形式で投稿します。今回は Platform engineering with Ama…

Introduction Agenda Data Integration challenges Streaming Data Technology Apache Flink for streaming analytics Apache Flink for breakin data silos Transaction Data Lake Summary IntroductionAWS Re:Invent の中で参加した Break down data silos…

Introduction What is Apache Iceberg ? Tabular on AWS Try Tabular Product Connection Iceberg Table IntroductionAWS Re:Invent のセッションの内容を受けて、実際に使ってみた内容をレポート形式で投稿します。今回は Tabular 社が登壇された LT 枠のセ…

Introduction Agenda Kubernetes for data processing Data on EKS(DoEKS) Project OSS data platforms on Amazon EKS Pinterest’s data modernization journey Best practices for Apache Spark at scale IntroductionAWS Re:Invent の中で参加したセッショ…

Introduction Challenges and Solutions Point to AWS Technology Iceberg to s3 Data Portal IntroductionAWS Re:Ivent に参加してます、セッションをレポート形式で投稿します。今回は Data Engineer 、IT Professional 向け、タイトル How AWS and BMW Gr…

Target Contents Event Infomation Dive Deep Session - Asset Management - Asset Management Pattern Summary Asset Management Pattern - Template - Asset Management Pattern - Asset As Code(AaC) - Asset Management Pattern - Asset As Bundle(AaB) …

Target Contents OverView The Azure Data Factory For Azure DevOps For Azure DevOps Pipelines Azure Resource Manager Azure Resource Manager Template Release Architecture the Azure Data Factory First Step Second Step Third Step Azure Data Fac…

Target Reader OverView The Amazon QuickSight Architecture DashBoard Development Process (開発プロセス) User Design (ユーザー設計) Access Design (アクセス設計) 1. Amazon QuickSight to AWS Resoruce Access (AWS リソースへのアクセス) 2. Amazon…