{"id":23311,"date":"2026-07-10T04:43:31","date_gmt":"2026-07-10T04:43:31","guid":{"rendered":"https:\/\/expertextend.de\/?p=23311"},"modified":"2026-07-16T06:42:39","modified_gmt":"2026-07-16T06:42:39","slug":"medallion-architecture-microsoft-fabric","status":"publish","type":"post","link":"https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/","title":{"rendered":"Medallion Architecture In Fabric"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"23311\" class=\"elementor elementor-23311\" data-elementor-post-type=\"post\">\n\t\t\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-61f082c e-con-full e-ecs-flex e-flex e-con e-parent\" data-id=\"61f082c\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;background_background&quot;:&quot;classic&quot;,&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-35ee82d e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"35ee82d\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-33dcce3 elementor-widget__width-initial elementor-widget elementor-widget-heading\" data-id=\"33dcce3\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h1 class=\"elementor-heading-title elementor-size-default\">Medallion Architecture in Microsoft Fabric: From Ingestion to Secured Reporting<\/h1>\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d882c98 elementor-widget__width-initial elementor-widget elementor-widget-text-editor\" data-id=\"d882c98\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p>A practical, end-to-end walkthrough of building the bronze, silver, and gold layers in Microsoft Fabric, including data masking and row-level security in the gold layer.<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2cb0089 elementor-widget-divider--view-line elementor-widget elementor-widget-divider\" data-id=\"2cb0089\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"divider.default\">\n\t\t\t\t\t\t\t<div class=\"elementor-divider\">\n\t\t\t<span class=\"elementor-divider-separator\">\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9758f7c elementor-widget__width-initial elementor-author-box--align-left elementor-widget elementor-widget-author-box\" data-id=\"9758f7c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"author-box.default\">\n\t\t\t\t\t\t\t<div class=\"elementor-author-box\">\n\t\t\t\t\t\t\t<div  class=\"elementor-author-box__avatar\">\n\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Gemini_Generated_Image_gt19i1gt19i1gt19-300x300.png\" alt=\"Picture of Sabhye Singhal&lt;br&gt;Lead Data Engineer, Expert-Extend \" loading=\"lazy\">\n\t\t\t\t<\/div>\n\t\t\t\n\t\t\t<div class=\"elementor-author-box__text\">\n\t\t\t\t\t\t\t\t\t<div >\n\t\t\t\t\t\t<h4 class=\"elementor-author-box__name\">\n\t\t\t\t\t\t\tSabhye Singhal<br>Lead Data Engineer, Expert-Extend \t\t\t\t\t\t<\/h4>\n\t\t\t\t\t<\/div>\n\t\t\t\t\n\t\t\t\t\n\t\t\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-eccdfe2 e-con-full e-ecs-flex e-flex e-con e-parent\" data-id=\"eccdfe2\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-00b4f36 e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"00b4f36\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-a4d83ae e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"a4d83ae\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;background_background&quot;:&quot;classic&quot;,&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-83ff7e1 elementor-widget elementor-widget-image\" data-id=\"83ff7e1\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img fetchpriority=\"high\" decoding=\"async\" width=\"1774\" height=\"887\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Medallion-Architecture-2.png\" class=\"attachment-full size-full wp-image-23332\" alt=\"\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Medallion-Architecture-2.png 1774w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Medallion-Architecture-2-300x150.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Medallion-Architecture-2-1024x512.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Medallion-Architecture-2-768x384.png 768w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Medallion-Architecture-2-1536x768.png 1536w\" sizes=\"(max-width: 1774px) 100vw, 1774px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Medallion architecture diagram in Microsoft Fabric: data sources flow via pipelines and shortcuts into Bronze, Silver and Gold layers in OneLake, then to Power BI and the SQL analytics endpoint.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-b522758 e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"b522758\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;background_background&quot;:&quot;classic&quot;,&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-622f4de elementor-toc--minimized-on-tablet elementor-widget elementor-widget-table-of-contents\" data-id=\"622f4de\" data-element_type=\"widget\" data-e-type=\"widget\" data-settings=\"{&quot;headings_by_tags&quot;:[&quot;h2&quot;,&quot;h3&quot;],&quot;exclude_headings_by_selector&quot;:[],&quot;no_headings_message&quot;:&quot;No headings were found on this page.&quot;,&quot;marker_view&quot;:&quot;numbers&quot;,&quot;minimize_box&quot;:&quot;yes&quot;,&quot;minimized_on&quot;:&quot;tablet&quot;,&quot;hierarchical_view&quot;:&quot;yes&quot;,&quot;min_height&quot;:{&quot;unit&quot;:&quot;px&quot;,&quot;size&quot;:&quot;&quot;,&quot;sizes&quot;:[]},&quot;min_height_tablet&quot;:{&quot;unit&quot;:&quot;px&quot;,&quot;size&quot;:&quot;&quot;,&quot;sizes&quot;:[]},&quot;min_height_mobile&quot;:{&quot;unit&quot;:&quot;px&quot;,&quot;size&quot;:&quot;&quot;,&quot;sizes&quot;:[]}}\" data-widget_type=\"table-of-contents.default\">\n\t\t\t\t\t\t\t<div class=\"elementor-toc__header\">\n\t\t\t<h4 class=\"elementor-toc__header-title\">\n\t\t\t\tTable of Contents\t\t\t<\/h4>\n\t\t\t\t\t\t\t<div class=\"elementor-toc__toggle-button elementor-toc__toggle-button--expand\" role=\"button\" tabindex=\"0\" aria-controls=\"elementor-toc__622f4de\" aria-expanded=\"true\" aria-label=\"Open table of contents\"><svg aria-hidden=\"true\" class=\"e-font-icon-svg e-fas-chevron-down\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M207.029 381.476L12.686 187.132c-9.373-9.373-9.373-24.569 0-33.941l22.667-22.667c9.357-9.357 24.522-9.375 33.901-.04L224 284.505l154.745-154.021c9.379-9.335 24.544-9.317 33.901.04l22.667 22.667c9.373 9.373 9.373 24.569 0 33.941L240.971 381.476c-9.373 9.372-24.569 9.372-33.942 0z\"><\/path><\/svg><\/div>\n\t\t\t\t<div class=\"elementor-toc__toggle-button elementor-toc__toggle-button--collapse\" role=\"button\" tabindex=\"0\" aria-controls=\"elementor-toc__622f4de\" aria-expanded=\"true\" aria-label=\"Close table of contents\"><svg aria-hidden=\"true\" class=\"e-font-icon-svg e-fas-chevron-up\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M240.971 130.524l194.343 194.343c9.373 9.373 9.373 24.569 0 33.941l-22.667 22.667c-9.357 9.357-24.522 9.375-33.901.04L224 227.495 69.255 381.516c-9.379 9.335-24.544 9.317-33.901-.04l-22.667-22.667c-9.373-9.373-9.373-24.569 0-33.941L207.03 130.525c9.372-9.373 24.568-9.373 33.941-.001z\"><\/path><\/svg><\/div>\n\t\t\t\t\t<\/div>\n\t\t<div id=\"elementor-toc__622f4de\" class=\"elementor-toc__body\">\n\t\t\t<div class=\"elementor-toc__spinner-container\">\n\t\t\t\t<svg class=\"elementor-toc__spinner eicon-animation-spin e-font-icon-svg e-eicon-loading\" aria-hidden=\"true\" viewBox=\"0 0 1000 1000\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M500 975V858C696 858 858 696 858 500S696 142 500 142 142 304 142 500H25C25 237 238 25 500 25S975 237 975 500 763 975 500 975Z\"><\/path><\/svg>\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b0dd82f elementor-widget elementor-widget-heading\" data-id=\"b0dd82f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">What Is Medallion Architecture?<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4627969 elementor-widget elementor-widget-text-editor\" data-id=\"4627969\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"TextRun SCXW194182640 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW194182640 BCX0\">The goal of medallion architecture is to incrementally improve the structure and quality of data. Think of medallion architecture as a three-stage cleaning and organizing process for your data. Each layer makes your data more reliable and easier to use.<\/span><\/span><\/p><p>1. <span class=\"TextRun SCXW54546403 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW54546403 BCX0\">Bronze (Raw): Store everything exactly as it arrives. No changes are allowed.<\/span><\/span><span class=\"EOP Selected SCXW54546403 BCX0\" data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:0,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p><p>2. <span class=\"TextRun SCXW171634605 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW171634605 BCX0\">Silver (Enriched): Fix errors, standardize formats, and remove duplicates.<\/span><\/span><\/p><p>3. <span class=\"TextRun SCXW154371310 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW154371310 BCX0\">Gold (Curated): Organize for reports and dashboards.<\/span><\/span><span class=\"EOP Selected SCXW154371310 BCX0\" data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:0,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p><p><span class=\"TextRun SCXW93240578 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW93240578 BCX0\">This architecture involves Extract, Transform<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">,<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">\u00a0and Load of the source data<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">. The source data can\u00a0<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">be<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">both internal<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">\u00a0(<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">One Lake<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">\u00a0or on-prem database<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">)<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">\u00a0or external<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">(<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">Azure Data Lake Store Gen2, Amazon S3, Google<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">Cloud Storage<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">, Azure SQL database<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">)<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">\u00a0storage systems<\/span><span class=\"NormalTextRun SCXW93240578 BCX0\">.<\/span><\/span><span class=\"EOP Selected SCXW93240578 BCX0\" data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:1,&quot;335551620&quot;:1,&quot;335559685&quot;:0,&quot;335559737&quot;:0,&quot;335559738&quot;:0,&quot;335559739&quot;:0,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-33c5fee elementor-widget elementor-widget-heading\" data-id=\"33c5fee\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">How Data Flows Through the Layers<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5adb8f5 elementor-widget elementor-widget-heading\" data-id=\"5adb8f5\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Extract: Source to Bronze<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-54f176c elementor-widget elementor-widget-text-editor\" data-id=\"54f176c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p>&#8211; <span class=\"TextRun SCXW254374242 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW254374242 BCX0\">Extract data from the source side to the <\/span><span class=\"NormalTextRun SCXW254374242 BCX0\">b<\/span><span class=\"NormalTextRun SCXW254374242 BCX0\">ronze layer <\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW254374242 BCX0\">Lakehouse<\/span><span class=\"NormalTextRun SCXW254374242 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW254374242 BCX0\">using shortcuts<\/span><span class=\"NormalTextRun SCXW254374242 BCX0\">\u00a0or through pipelines using copy data activity<\/span><span class=\"NormalTextRun SCXW254374242 BCX0\">\u00a0and save it\u00a0<\/span><span class=\"NormalTextRun SCXW254374242 BCX0\">in<\/span><span class=\"NormalTextRun SCXW254374242 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW254374242 BCX0\">delta format<\/span><span class=\"NormalTextRun SCXW254374242 BCX0\">.\u00a0<\/span><\/span><span class=\"EOP Selected SCXW254374242 BCX0\" data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:0,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p><p>&#8211; <span class=\"TextRun SCXW92684865 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW92684865 BCX0\">In the bronze layer, you store data in its original format, which might be either tables or files.<\/span><span class=\"NormalTextRun SCXW92684865 BCX0\">\u00a0Tables will be used to store structured datasets<\/span><span class=\"NormalTextRun SCXW92684865 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW92684865 BCX0\">(CSV, Parquet, or Delta)<\/span><span class=\"NormalTextRun SCXW92684865 BCX0\">,<\/span><span class=\"NormalTextRun SCXW92684865 BCX0\">\u00a0and files will be used to store sem<\/span><span class=\"NormalTextRun SCXW92684865 BCX0\">i<\/span><span class=\"NormalTextRun SCXW92684865 BCX0\">-structured or unstructured datasets<\/span><span class=\"NormalTextRun SCXW92684865 BCX0\">\u00a0(JSON<\/span><span class=\"NormalTextRun SCXW92684865 BCX0\">s,\u00a0<\/span><span class=\"NormalTextRun SCXW92684865 BCX0\">logs,\u00a0<\/span><span class=\"NormalTextRun SCXW92684865 BCX0\">PDF<\/span><span class=\"NormalTextRun SCXW92684865 BCX0\">s<\/span><span class=\"NormalTextRun SCXW92684865 BCX0\">, images, IOT readings<\/span><span class=\"NormalTextRun SCXW92684865 BCX0\">)<\/span><span class=\"NormalTextRun SCXW92684865 BCX0\">.<\/span><\/span><span class=\"EOP Selected SCXW92684865 BCX0\" data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:0,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p><p>&#8211; <span class=\"TextRun SCXW67141738 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW67141738 BCX0\">In the bronze layer, you can have smaller files because of the raw nature of the data, as long you focus data modification and preparation with Spark.<\/span><\/span><span class=\"EOP Selected SCXW67141738 BCX0\" data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:0,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-87d693a elementor-widget elementor-widget-heading\" data-id=\"87d693a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\"> Transform: Bronze to Silver<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-64de087 elementor-widget elementor-widget-text-editor\" data-id=\"64de087\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p>&#8211; <span class=\"TextRun SCXW220936088 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW220936088 BCX0\">Transform the data from Bronze\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW220936088 BCX0\">lakehouse<\/span><span class=\"NormalTextRun SCXW220936088 BCX0\"> to Silver\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW220936088 BCX0\">lakehouse,<\/span><span class=\"NormalTextRun SCXW220936088 BCX0\">\u00a0either using a notebook or Dataflow Gen2.<\/span><\/span><span class=\"EOP Selected SCXW220936088 BCX0\" data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:0,&quot;335559740&quot;:240}\">\u00a0<\/span><span class=\"EOP Selected SCXW254374242 BCX0\" data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:0,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p><p>&#8211; <span class=\"TextRun SCXW22639389 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW22639389 BCX0\">In Silver layer, you store the data as delta tables<\/span><span class=\"NormalTextRun SCXW22639389 BCX0\">\u00a0after applying transformation rules such as\u00a0<\/span><span class=\"NormalTextRun SCXW22639389 BCX0\">data quality checks, null checks, removing duplicate records,\u00a0<\/span><span class=\"NormalTextRun SCXW22639389 BCX0\">appropriate aggregations<\/span><span class=\"NormalTextRun SCXW22639389 BCX0\"> as per the business requirements,<\/span><span class=\"NormalTextRun SCXW22639389 BCX0\">\u00a0etc.<\/span><\/span><span class=\"EOP Selected SCXW22639389 BCX0\" data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:0,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p><p>&#8211; <span class=\"TextRun SCXW221707495 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW221707495 BCX0\">Delta tables use<\/span><span class=\"NormalTextRun SCXW221707495 BCX0\">\u00a0V-Order write-time optimization to the Parquet file format. That optimization enables fast reads by Fabric compute engines, such as Power BI, SQL, Apache Spark, and others.<\/span><\/span><span class=\"EOP Selected SCXW221707495 BCX0\" data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:0,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0abd15c elementor-widget elementor-widget-heading\" data-id=\"0abd15c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Load: Silver to Gold<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6dfaa38 elementor-widget elementor-widget-text-editor\" data-id=\"6dfaa38\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p>&#8211; <span class=\"TextRun SCXW8111491 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW8111491 BCX0\">Once the data is transformed, we will load the data into the gold layer.<\/span><\/span><span class=\"EOP Selected SCXW8111491 BCX0\" data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:0,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p><p>&#8211; This layer will be a warehouse, unlike bronze and silver layers, which were lakehouses.<\/p><p>&#8211; <span class=\"TextRun SCXW236133534 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW236133534 BCX0\">Data gets loaded to the gold layer <\/span><span class=\"NormalTextRun SCXW236133534 BCX0\">by either using a pipeline or through dataflow Gen2.<\/span><\/span><span class=\"EOP Selected SCXW236133534 BCX0\" data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:0,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p><p>&#8211; The gold<span class=\"NormalTextRun SCXW145714152 BCX0\">\u00a0layer will be the\u00a0<\/span><span class=\"NormalTextRun SCXW145714152 BCX0\">SQ<\/span><span class=\"NormalTextRun SCXW145714152 BCX0\">L<\/span><span class=\"NormalTextRun SCXW145714152 BCX0\">\u00a0analytics\u00a0<\/span><span class=\"NormalTextRun SCXW145714152 BCX0\">endpoint for the semantic models or the reports<\/span><span class=\"NormalTextRun SCXW145714152 BCX0\">\u00a0in our workspace<\/span><span class=\"NormalTextRun SCXW145714152 BCX0\">.<\/span><\/p><p>&#8211; The gold<span class=\"NormalTextRun SCXW182674221 BCX0\">\u00a0layer will\u00a0<\/span><span class=\"NormalTextRun SCXW182674221 BCX0\">contain<\/span><span class=\"NormalTextRun SCXW182674221 BCX0\"> the\u00a0<\/span><span class=\"NormalTextRun SCXW182674221 BCX0\">Star schema<\/span><span class=\"NormalTextRun SCXW182674221 BCX0\">\u00a0fact and dimension tables and any relationships<\/span><span class=\"NormalTextRun SCXW182674221 BCX0\">\u00a0in between them<\/span><span class=\"NormalTextRun SCXW182674221 BCX0\">.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-36ece4c elementor-widget elementor-widget-heading\" data-id=\"36ece4c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Purpose of the blog<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e3223f9 elementor-widget elementor-widget-text-editor\" data-id=\"e3223f9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p>&#8211; <span class=\"TextRun SCXW197423712 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW197423712 BCX0\">Technical guide on h<\/span><span class=\"NormalTextRun SCXW197423712 BCX0\">ow to implement Medallion architecture in Microsoft Fabric.<\/span><\/span><span class=\"EOP Selected SCXW197423712 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><span class=\"EOP Selected SCXW8111491 BCX0\" data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335559739&quot;:0,&quot;335559740&quot;:240}\">\u00a0<\/span><\/p><p>&#8211; <span class=\"TextRun SCXW24644799 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW24644799 BCX0\">Technical guide on h<\/span><span class=\"NormalTextRun SCXW24644799 BCX0\">ow to implement\u00a0<\/span><span class=\"NormalTextRun SCXW24644799 BCX0\">data masking and row-level security<\/span><span class=\"NormalTextRun SCXW24644799 BCX0\"> in the gold layer.<\/span><\/span><span class=\"EOP Selected SCXW24644799 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/p><p>&#8211; <span class=\"NormalTextRun SCXW139194261 BCX0\">Identification of\u00a0<\/span><span class=\"NormalTextRun SCXW139194261 BCX0\">the\u00a0<\/span><span class=\"NormalTextRun SCXW139194261 BCX0\">limit<\/span><span class=\"NormalTextRun SCXW139194261 BCX0\">a<\/span><span class=\"NormalTextRun SCXW139194261 BCX0\">tions\u00a0<\/span><span class=\"NormalTextRun SCXW139194261 BCX0\">in<\/span><span class=\"NormalTextRun SCXW139194261 BCX0\"> the\u00a0<\/span><span class=\"NormalTextRun SCXW139194261 BCX0\">previous<\/span><span class=\"NormalTextRun SCXW139194261 BCX0\">\u00a0architecture and<\/span><span class=\"NormalTextRun SCXW139194261 BCX0\"> the\u00a0<\/span><span class=\"NormalTextRun SCXW139194261 BCX0\">advantages of the current ar<\/span><span class=\"NormalTextRun SCXW139194261 BCX0\">chitecture.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-89048bc elementor-widget elementor-widget-heading\" data-id=\"89048bc\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Implementing Medallion Architecture in Microsoft Fabric, Step by Step<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-66c08a4 elementor-widget elementor-widget-heading\" data-id=\"66c08a4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Building the Bronze Layer (Raw Data)<\/h3>\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-2104fbb e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"2104fbb\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-562318e elementor-widget elementor-widget-text-editor\" data-id=\"562318e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"TextRun SCXW113824529 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW113824529 BCX0\">&#8211; This layer <\/span><span class=\"NormalTextRun SCXW113824529 BCX0\">contains<\/span><span class=\"NormalTextRun SCXW113824529 BCX0\">\u00a0a raw data\u00a0<\/span><span class=\"NormalTextRun SCXW113824529 BCX0\">or a copy of the data from the source side<\/span><span class=\"NormalTextRun SCXW113824529 BCX0\">\u00a0without any kind of transformations applied<\/span><span class=\"NormalTextRun SCXW113824529 BCX0\">.<\/span><\/span><span class=\"EOP Selected SCXW113824529 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-5107c1c e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"5107c1c\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-af90f9e elementor-widget elementor-widget-text-editor\" data-id=\"af90f9e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW137820166 BCX0\">&#8211; Assuming we have already<\/span><span class=\"NormalTextRun SCXW137820166 BCX0\">\u00a0created a workspace, click on\u00a0<\/span><span class=\"NormalTextRun SCXW137820166 BCX0\">new item<\/span><span class=\"NormalTextRun SCXW137820166 BCX0\">\u00a0as shown below<\/span><span class=\"NormalTextRun SCXW137820166 BCX0\">.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-04ddd21 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"04ddd21\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"752\" height=\"374\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture-1.png\" class=\"attachment-full size-full wp-image-23405\" alt=\"Microsoft Fabric workspace with the New item button selected to create a lakehouse.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture-1.png 752w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture-1-300x149.png 300w\" sizes=\"(max-width: 752px) 100vw, 752px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\"> Microsoft Fabric workspace with the New item button selected to create a lakehouse.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-c86a7e4 e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"c86a7e4\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-c268c60 elementor-widget elementor-widget-text-editor\" data-id=\"c268c60\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW137820166 BCX0\">&#8211; <span class=\"NormalTextRun SCXW127779590 BCX0\">Search for Lakehouse item and click on that item<\/span><span class=\"NormalTextRun SCXW127779590 BCX0\">.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-060c0a1 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"060c0a1\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"1379\" height=\"465\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture2.png\" class=\"attachment-full size-full wp-image-23404\" alt=\"Fabric New item panel with Lakehouse selected from the item type list.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture2.png 1379w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture2-300x101.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture2-1024x345.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture2-768x259.png 768w\" sizes=\"(max-width: 1379px) 100vw, 1379px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Fabric New item panel with Lakehouse selected from the item type list.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-a6d3398 e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"a6d3398\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-6b72b5f elementor-widget elementor-widget-text-editor\" data-id=\"6b72b5f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW137820166 BCX0\">&#8211; <span class=\"TextRun SCXW68628301 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW68628301 BCX0\">Give the name as Bronze Layer<\/span><span class=\"NormalTextRun SCXW68628301 BCX0\">, provide the location<\/span><span class=\"NormalTextRun SCXW68628301 BCX0\"> where you want to save the item in the workspace, <\/span><span class=\"NormalTextRun SCXW68628301 BCX0\">check\/<\/span><span class=\"NormalTextRun SCXW68628301 BCX0\">uncheck Lakehouse Schemas<\/span><span class=\"NormalTextRun SCXW68628301 BCX0\"> (Schemas let you manage and reference objects in your <\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW68628301 BCX0\">lakehouse<\/span><span class=\"NormalTextRun SCXW68628301 BCX0\">\u00a0more easily. You can use a default schema to automatically fill in file paths when referencing a data object in code) depending upon the use case<\/span><span class=\"NormalTextRun SCXW68628301 BCX0\">\u00a0and click on\u00a0<\/span><\/span><span class=\"TextRun SCXW68628301 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW68628301 BCX0\">Create<\/span><\/span><span class=\"TextRun SCXW68628301 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW68628301 BCX0\">.<\/span><span class=\"NormalTextRun SCXW68628301 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW68628301 BCX0\">You will see\u00a0<\/span><span class=\"NormalTextRun SCXW68628301 BCX0\">a new\u00a0<\/span><span class=\"NormalTextRun SCXW68628301 BCX0\">item<\/span><span class=\"NormalTextRun SCXW68628301 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW68628301 BCX0\">of Lakehouse being created.<\/span><\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ae99819 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"ae99819\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1203\" height=\"677\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture3.png\" class=\"attachment-full size-full wp-image-23403\" alt=\"Create lakehouse dialog with the name set to Bronze Layer and the Lakehouse schemas option shown.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture3.png 1203w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture3-300x169.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture3-1024x576.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture3-768x432.png 768w\" sizes=\"(max-width: 1203px) 100vw, 1203px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Create lakehouse dialog with the name set to Bronze Layer and the Lakehouse schemas option shown.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-632bfec e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"632bfec\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-ccbbd04 elementor-widget elementor-widget-text-editor\" data-id=\"ccbbd04\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW137820166 BCX0\">&#8211; <span class=\"TextRun SCXW28023560 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW28023560 BCX0\">Go inside that item and click on the <\/span><span class=\"NormalTextRun SCXW28023560 BCX0\">three dots beside the <\/span><span class=\"NormalTextRun SCXW28023560 BCX0\">Files Folder.<\/span><span class=\"NormalTextRun SCXW28023560 BCX0\">\u00a0<strong>Click on\u00a0<\/strong><\/span><\/span><strong><span class=\"TextRun SCXW28023560 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW28023560 BCX0\">New Shortcut<\/span><\/span><span class=\"TextRun SCXW28023560 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW28023560 BCX0\">.<\/span><\/span><\/strong><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-90499bf elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"90499bf\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1204\" height=\"816\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture5.png\" class=\"attachment-full size-full wp-image-23401\" alt=\"New shortcut data source screen with a SharePoint folder selected as the source.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture5.png 1204w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture5-300x203.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture5-1024x694.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture5-768x521.png 768w\" sizes=\"(max-width: 1204px) 100vw, 1204px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Bronze Layer lakehouse Files folder with the three-dot menu open on the New shortcut option.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-2bcf515 e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"2bcf515\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-f218c1c elementor-widget elementor-widget-text-editor\" data-id=\"f218c1c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW137820166 BCX0\">&#8211; <span class=\"TextRun SCXW28023560 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW28023560 BCX0\">Go inside that item and click on the <\/span><span class=\"NormalTextRun SCXW28023560 BCX0\">three dots beside the <\/span><span class=\"NormalTextRun SCXW28023560 BCX0\">Files Folder.<\/span><span class=\"NormalTextRun SCXW28023560 BCX0\">\u00a0<strong>Click on\u00a0<\/strong><\/span><\/span><strong><span class=\"TextRun SCXW28023560 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW28023560 BCX0\">New Shortcut<\/span><\/span><span class=\"TextRun SCXW28023560 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW28023560 BCX0\">.<\/span><\/span><\/strong><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-375dd8d elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"375dd8d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1204\" height=\"816\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture5.png\" class=\"attachment-full size-full wp-image-23401\" alt=\"New shortcut data source screen with a SharePoint folder selected as the source.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture5.png 1204w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture5-300x203.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture5-1024x694.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture5-768x521.png 768w\" sizes=\"(max-width: 1204px) 100vw, 1204px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">New shortcut data source screen with a SharePoint folder selected as the source.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-287c884 e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"287c884\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-5bf9cac elementor-widget elementor-widget-text-editor\" data-id=\"5bf9cac\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW137820166 BCX0\">&#8211; <span class=\"TextRun SCXW258270614 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun ContextualSpellingAndGrammarErrorV2Themed SCXW258270614 BCX0\">Since<\/span><span class=\"NormalTextRun SCXW258270614 BCX0\">\u00a0initially we\u00a0<\/span><span class=\"NormalTextRun SCXW258270614 BCX0\">won\u2019t<\/span><span class=\"NormalTextRun SCXW258270614 BCX0\"> have an existing connection, we will create a new connection and provide the authentication<\/span><span class=\"NormalTextRun SCXW258270614 BCX0\">\u00a0kind as Organizational account.<\/span><span class=\"NormalTextRun SCXW258270614 BCX0\"> Click on the\u00a0<\/span><\/span><strong><span class=\"TextRun SCXW258270614 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW258270614 BCX0\">Next<\/span><\/span><\/strong><span class=\"TextRun SCXW258270614 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW258270614 BCX0\"><strong>\u00a0button<\/strong>.<\/span><\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1cd957a elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"1cd957a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1377\" height=\"800\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture6.png\" class=\"attachment-full size-full wp-image-23400\" alt=\"Shortcut connection setup with authentication kind set to Organizational account.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture6.png 1377w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture6-300x174.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture6-1024x595.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture6-768x446.png 768w\" sizes=\"(max-width: 1377px) 100vw, 1377px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Shortcut connection setup with authentication kind set to Organizational account.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-c785094 e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"c785094\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-495cf98 elementor-widget elementor-widget-text-editor\" data-id=\"495cf98\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW137820166 BCX0\">&#8211; <span class=\"TextRun SCXW15758193 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW15758193 BCX0\">Select\u00a0<\/span><span class=\"NormalTextRun SCXW15758193 BCX0\">an appropriate directory<\/span><span class=\"NormalTextRun SCXW15758193 BCX0\">\u00a0of which you want to create a shortcut.<\/span><span class=\"NormalTextRun SCXW15758193 BCX0\">\u00a0Click on Next button.<\/span><\/span><span class=\"EOP Selected SCXW15758193 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c97bde9 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"c97bde9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1203\" height=\"669\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture7.png\" class=\"attachment-full size-full wp-image-23399\" alt=\"Shortcut wizard step for selecting the source directory to shortcut into the lakehouse.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture7.png 1203w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture7-300x167.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture7-1024x569.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture7-768x427.png 768w\" sizes=\"(max-width: 1203px) 100vw, 1203px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Shortcut wizard step for selecting the source directory to shortcut into the lakehouse.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-31b39c4 e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"31b39c4\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-64de667 elementor-widget elementor-widget-text-editor\" data-id=\"64de667\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW137820166 BCX0\">&#8211; <span class=\"TextRun SCXW150129014 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW150129014 BCX0\">It will summarize with the location of the shortcut. Click on Create button.<\/span><\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a8632f0 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"a8632f0\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1379\" height=\"768\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture8.png\" class=\"attachment-full size-full wp-image-23398\" alt=\"Shortcut summary screen showing the target location before creating the shortcut.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture8.png 1379w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture8-300x167.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture8-1024x570.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture8-768x428.png 768w\" sizes=\"(max-width: 1379px) 100vw, 1379px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Shortcut summary screen showing the target location before creating the shortcut.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-4a75246 e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"4a75246\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-38cadea elementor-widget elementor-widget-text-editor\" data-id=\"38cadea\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">&#8211; A new shortcut will be created with the same name as the directory name at the source side. In our case, we have already made a shortcut with a different directory as the source, hence it won\u2019t be seen in the image below.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4a24b8c elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"4a24b8c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1379\" height=\"224\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture9.png\" class=\"attachment-full size-full wp-image-23397\" alt=\"Bronze Layer lakehouse Files folder showing the newly created shortcut to the source directory.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture9.png 1379w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture9-300x49.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture9-1024x166.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture9-768x125.png 768w\" sizes=\"(max-width: 1379px) 100vw, 1379px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Bronze Layer lakehouse Files folder showing the newly created shortcut to the source directory.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c5a3383 elementor-widget elementor-widget-text-editor\" data-id=\"c5a3383\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">&#8211; <span class=\"NormalTextRun SCXW119855912 BCX0\">Similarly,<\/span><span class=\"NormalTextRun SCXW119855912 BCX0\">\u00a0we can do for other\u00a0<\/span><span class=\"NormalTextRun SCXW119855912 BCX0\">data\u00a0<\/span><span class=\"NormalTextRun SCXW119855912 BCX0\">source<\/span><span class=\"NormalTextRun SCXW119855912 BCX0\">s<\/span><span class=\"NormalTextRun SCXW119855912 BCX0\">\u00a0as well.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c2cc641 elementor-widget elementor-widget-heading\" data-id=\"c2cc641\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Building the Silver Layer (Transformed Data)<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-71e87cc elementor-widget elementor-widget-text-editor\" data-id=\"71e87cc\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"TextRun SCXW125345088 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW125345088 BCX0\">&#8211; This layer <\/span><span class=\"NormalTextRun SCXW125345088 BCX0\">contains <\/span><\/span><span class=\"TextRun SCXW125345088 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW125345088 BCX0\">transformed <\/span><\/span><span class=\"TextRun SCXW125345088 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW125345088 BCX0\">data. This means it will have the data after implementing <\/span><span class=\"NormalTextRun SCXW125345088 BCX0\">all kinds of transformations, such as <\/span><\/span><span class=\"TextRun SCXW125345088 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW125345088 BCX0\">data quality checks, aggregations, joins, unions,\u00a0<\/span><span class=\"NormalTextRun SCXW125345088 BCX0\">data type checks, null checks,<\/span><\/span><span class=\"TextRun SCXW125345088 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW125345088 BCX0\">\u00a0etc.<\/span><\/span><span class=\"EOP Selected SCXW125345088 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/p><p>&#8211; <span class=\"NormalTextRun SCXW265623961 BCX0\">To transform the data, we can\u00a0<\/span><span class=\"NormalTextRun SCXW265623961 BCX0\">either use a Spark Notebook or a <\/span><span class=\"NormalTextRun SCXW265623961 BCX0\">D<\/span><span class=\"NormalTextRun SCXW265623961 BCX0\">ataflow <\/span><span class=\"NormalTextRun SCXW265623961 BCX0\">Gen2.<\/span><span class=\"NormalTextRun SCXW265623961 BCX0\">\u00a0In our case, we are creating a\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW265623961 BCX0\">PySpark<\/span><span class=\"NormalTextRun SCXW265623961 BCX0\">\u00a0Notebook.<\/span><\/p><p>&#8211; <span data-contrast=\"auto\">To create this layer, again click on\u00a0new item\u00a0as shown above in Bronze Layer.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">&#8211; Search for the Lakehouse item and click on that item.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">&#8211; Give the name as Silver Layer, provide the location where you want to save the item in the workspace, check\/uncheck Lakehouse Schemas as per the use case, and click on <\/span><b><span data-contrast=\"auto\">Create<\/span><\/b><span data-contrast=\"auto\">. You will see\u00a0a new item\u00a0of Lakehouse being created.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">&#8211; Now, again click on the new item as shown above in the Bronze Layer.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-cf4ffc4 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"cf4ffc4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1203\" height=\"348\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture10.png\" class=\"attachment-full size-full wp-image-23396\" alt=\"Fabric New item search showing the Notebook option for the Silver layer\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture10.png 1203w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture10-300x87.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture10-1024x296.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture10-768x222.png 768w\" sizes=\"(max-width: 1203px) 100vw, 1203px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Fabric New item search showing the Notebook option for the Silver layer<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-3a9c660 e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"3a9c660\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-abb05c9 elementor-widget elementor-widget-text-editor\" data-id=\"abb05c9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">&#8211; <span class=\"NormalTextRun SCXW66186694 BCX0\">Search for Notebook<\/span><span class=\"NormalTextRun SCXW66186694 BCX0\">\/Dataflow Gen2<\/span><span class=\"NormalTextRun SCXW66186694 BCX0\">.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9f8538e elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"9f8538e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1204\" height=\"341\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture11.png\" class=\"attachment-full size-full wp-image-23395\" alt=\"Fabric New item search showing the Dataflow Gen2 option for the Silver layer transformation.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture11.png 1204w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture11-300x85.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture11-1024x290.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture11-768x218.png 768w\" sizes=\"(max-width: 1204px) 100vw, 1204px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Fabric New item search showing the Dataflow Gen2 option for the Silver layer transformation.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-5245cb5 e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"5245cb5\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-0436b4a elementor-widget elementor-widget-text-editor\" data-id=\"0436b4a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><strong>For Notebook:<\/strong><\/p><p><span class=\"NormalTextRun SCXW113931281 BCX0\">&#8211; Enter the <\/span><span class=\"NormalTextRun SCXW113931281 BCX0\">name of the item and the location where you want to save<\/span><span class=\"NormalTextRun SCXW113931281 BCX0\">.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-16c8212 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"16c8212\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1054\" height=\"462\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture12.png\" class=\"attachment-full size-full wp-image-23394\" alt=\"New notebook dialog with fields for the notebook name and save location.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture12.png 1054w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture12-300x131.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture12-1024x449.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture12-768x337.png 768w\" sizes=\"(max-width: 1054px) 100vw, 1054px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Fabric New item search showing the Dataflow Gen2 option for the Silver layer transformation.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-282b523 elementor-widget elementor-widget-text-editor\" data-id=\"282b523\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW113931281 BCX0\">&#8211; <span class=\"TextRun SCXW177826154 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW177826154 BCX0\">Click on\u00a0<\/span><\/span><span class=\"TextRun SCXW177826154 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW177826154 BCX0\">Create<\/span><\/span><span class=\"TextRun SCXW177826154 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW177826154 BCX0\">\u00a0button.<\/span><\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-0f5da73 e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"0f5da73\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-0abc312 elementor-widget elementor-widget-text-editor\" data-id=\"0abc312\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><strong><span class=\"TextRun SCXW149240833 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW149240833 BCX0\">For Dataflow Gen2<\/span><\/span>:<\/strong><\/p><p><span class=\"NormalTextRun SCXW113931281 BCX0\">&#8211; <span class=\"NormalTextRun SCXW188413608 BCX0\">Enter the name of the dataflow and uncheck\u00a0<\/span><span class=\"NormalTextRun SCXW188413608 BCX0\">Git integration box.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9249092 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"9249092\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"846\" height=\"558\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture13.png\" class=\"attachment-full size-full wp-image-23393\" alt=\"New Dataflow Gen2 dialog with the Git integration checkbox unchecked.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture13.png 846w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture13-300x198.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture13-768x507.png 768w\" sizes=\"(max-width: 846px) 100vw, 846px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">New Dataflow Gen2 dialog with the Git integration checkbox unchecked.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-50a606a elementor-widget elementor-widget-text-editor\" data-id=\"50a606a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW113931281 BCX0\">&#8211; <span class=\"TextRun SCXW177826154 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW177826154 BCX0\">Click on\u00a0<\/span><\/span><span class=\"TextRun SCXW177826154 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW177826154 BCX0\">Create<\/span><\/span><span class=\"TextRun SCXW177826154 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW177826154 BCX0\">\u00a0button.<\/span><\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-005e32b e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"005e32b\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-ac4c76d elementor-widget elementor-widget-text-editor\" data-id=\"ac4c76d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW176063908 BCX0\">&#8211; You will see <\/span><span class=\"NormalTextRun SCXW176063908 BCX0\">a new item<\/span><span class=\"NormalTextRun SCXW176063908 BCX0\">\u00a0of Notebook\/Dataflow Gen2 being created.<\/span><\/p><p>&#8211; Open the item<\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-4771f2c e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"4771f2c\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-80543b9 elementor-widget elementor-widget-text-editor\" data-id=\"80543b9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><strong><span class=\"TextRun SCXW149240833 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW149240833 BCX0\">For Notebook<\/span><\/span>:<\/strong><\/p><p><span class=\"NormalTextRun SCXW113931281 BCX0\">&#8211; Click on Data items<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1d05f23 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"1d05f23\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1203\" height=\"248\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture14.png\" class=\"attachment-full size-full wp-image-23392\" alt=\"Fabric notebook editor with the Data items button selected to attach lakehouses.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture14.png 1203w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture14-300x62.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture14-1024x211.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture14-768x158.png 768w\" sizes=\"(max-width: 1203px) 100vw, 1203px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Fabric notebook editor with the Data items button selected to attach lakehouses.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a20bc3f elementor-widget elementor-widget-text-editor\" data-id=\"a20bc3f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW113931281 BCX0\">&#8211; <span class=\"TextRun SCXW177826154 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW177826154 BCX0\">Click on the OneLake Catalog<\/span><\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8afd6a0 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"8afd6a0\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"828\" height=\"558\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture15.png\" class=\"attachment-full size-full wp-image-23391\" alt=\"OneLake catalog dialog for selecting lakehouses to load into the notebook.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture15.png 828w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture15-300x202.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture15-768x518.png 768w\" sizes=\"(max-width: 828px) 100vw, 828px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">OneLake catalog dialog for selecting lakehouses to load into the notebook.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7bd583b elementor-widget elementor-widget-text-editor\" data-id=\"7bd583b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW113931281 BCX0\">&#8211; <span class=\"NormalTextRun SCXW94472778 BCX0\">Choos<\/span><span class=\"NormalTextRun SCXW94472778 BCX0\">e<\/span><span class=\"NormalTextRun SCXW94472778 BCX0\">\u00a0the Bronze and Silver Layer\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW94472778 BCX0\">lakehouse<\/span><span class=\"NormalTextRun SCXW94472778 BCX0\"> as the data items to load to the Notebook.<\/span><span class=\"NormalTextRun SCXW94472778 BCX0\"> Click on the Add button.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-72aeff8 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"72aeff8\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1203\" height=\"406\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture16.png\" class=\"attachment-full size-full wp-image-23390\" alt=\"OneLake catalog with the Bronze Layer and Silver Layer lakehouses selected to add to the notebook.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture16.png 1203w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture16-300x101.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture16-1024x346.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture16-768x259.png 768w\" sizes=\"(max-width: 1203px) 100vw, 1203px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">OneLake catalog with the Bronze Layer and Silver Layer lakehouses selected to add to the notebook.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-648ea8c elementor-widget elementor-widget-text-editor\" data-id=\"648ea8c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW113931281 BCX0\">&#8211; <span class=\"NormalTextRun SCXW213607380 BCX0\">Yo<\/span><span class=\"NormalTextRun SCXW213607380 BCX0\">u will see the data items as shown below added. Then, c<\/span><span class=\"NormalTextRun SCXW213607380 BCX0\">lick on connect<\/span><span class=\"NormalTextRun SCXW213607380 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW213607380 BCX0\">tab<\/span><span class=\"NormalTextRun SCXW213607380 BCX0\">\u00a0and\u00a0<\/span><span class=\"NormalTextRun SCXW213607380 BCX0\">on New Standard Session<\/span><span class=\"NormalTextRun SCXW213607380 BCX0\">\u00a0to connect to processing engine where the script will run<\/span><span class=\"NormalTextRun SCXW213607380 BCX0\">.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-cae103e elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"cae103e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"910\" height=\"568\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture17.png\" class=\"attachment-full size-full wp-image-23389\" alt=\"Notebook with Bronze and Silver lakehouses attached and the New standard session connect option.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture17.png 910w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture17-300x187.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture17-768x479.png 768w\" sizes=\"(max-width: 910px) 100vw, 910px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Notebook with Bronze and Silver lakehouses attached and the New standard session connect option.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7b0bdc9 elementor-widget elementor-widget-text-editor\" data-id=\"7b0bdc9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW113931281 BCX0\">&#8211; <span class=\"NormalTextRun SCXW80860734 BCX0\">Write the commands\u00a0<\/span><span class=\"NormalTextRun SCXW80860734 BCX0\">of loading the data from bronze layer, transform<\/span><span class=\"NormalTextRun SCXW80860734 BCX0\">ing the data<\/span><span class=\"NormalTextRun SCXW80860734 BCX0\">\u00a0on top of the loaded\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW80860734 BCX0\">dataframe<\/span><span class=\"NormalTextRun SCXW80860734 BCX0\">\u00a0<\/span><span class=\"NormalTextRun ContextualSpellingAndGrammarErrorV2Themed SCXW80860734 BCX0\">and\u00a0<\/span><span class=\"NormalTextRun ContextualSpellingAndGrammarErrorV2Themed SCXW80860734 BCX0\">\u00a0<\/span><span class=\"NormalTextRun ContextualSpellingAndGrammarErrorV2Themed SCXW80860734 BCX0\">writing<\/span><span class=\"NormalTextRun SCXW80860734 BCX0\">\u00a0the\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW80860734 BCX0\">dataframe<\/span><span class=\"NormalTextRun SCXW80860734 BCX0\">\u00a0to Silver Layer<\/span><span class=\"NormalTextRun SCXW80860734 BCX0\">\u00a0in the notebook cells<\/span><span class=\"NormalTextRun SCXW80860734 BCX0\">\u00a0as shown<\/span><span class=\"NormalTextRun SCXW80860734 BCX0\">.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3d58c53 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"3d58c53\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1203\" height=\"402\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture18.png\" class=\"attachment-full size-full wp-image-23388\" alt=\"PySpark notebook cell reading CSV data from the Bronze lakehouse into a dataframe, with a table preview (sensitive fields redacted).\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture18.png 1203w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture18-300x100.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture18-1024x342.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture18-768x257.png 768w\" sizes=\"(max-width: 1203px) 100vw, 1203px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">PySpark notebook cell reading CSV data from the Bronze lakehouse into a dataframe, with a table preview (sensitive fields redacted).<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a1405b7 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"a1405b7\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1204\" height=\"304\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture19.png\" class=\"attachment-full size-full wp-image-23387\" alt=\"PySpark notebook cell looping through columns to convert date fields from string to timestamp format.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture19.png 1204w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture19-300x76.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture19-1024x259.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture19-768x194.png 768w\" sizes=\"(max-width: 1204px) 100vw, 1204px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">PySpark notebook cell looping through columns to convert date fields from string to timestamp format.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ddf8bcc elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"ddf8bcc\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1204\" height=\"320\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture20.png\" class=\"attachment-full size-full wp-image-23386\" alt=\"PySpark notebook cell writing the transformed dataframe to the Silver layer as a Delta table in overwrite mode.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture20.png 1204w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture20-300x80.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture20-1024x272.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture20-768x204.png 768w\" sizes=\"(max-width: 1204px) 100vw, 1204px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">PySpark notebook cell writing the transformed dataframe to the Silver layer as a Delta table in overwrite mode.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-7b6cd56 e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"7b6cd56\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-f298842 elementor-widget elementor-widget-text-editor\" data-id=\"f298842\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><strong><span class=\"TextRun SCXW149240833 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW149240833 BCX0\">For Dataflow Gen2<\/span><\/span>:<\/strong><\/p><p><span class=\"NormalTextRun SCXW113931281 BCX0\">&#8211; <span class=\"TextRun SCXW265812148 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW265812148 BCX0\">Click on Get data<\/span><span class=\"NormalTextRun SCXW265812148 BCX0\">.<\/span><span class=\"NormalTextRun SCXW265812148 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW265812148 BCX0\">Click on\u00a0<\/span><span class=\"NormalTextRun SCXW265812148 BCX0\">More<\/span><span class=\"NormalTextRun SCXW265812148 BCX0\">.<\/span><\/span><span class=\"EOP Selected SCXW265812148 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4a54db9 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"4a54db9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"522\" height=\"888\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture21.png\" class=\"attachment-full size-full wp-image-23385\" alt=\"Dataflow Gen2 Get data screen with the More option selected.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture21.png 522w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture21-176x300.png 176w\" sizes=\"(max-width: 522px) 100vw, 522px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Dataflow Gen2 Get data screen with the More option selected.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-cfa1407 elementor-widget elementor-widget-text-editor\" data-id=\"cfa1407\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW113931281 BCX0\">&#8211; <span class=\"NormalTextRun SCXW153772824 BCX0\">Click on\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW153772824 BCX0\">OneLake<\/span><span class=\"NormalTextRun SCXW153772824 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW153772824 BCX0\">catalog<\/span><span class=\"NormalTextRun SCXW153772824 BCX0\">.<\/span><span class=\"NormalTextRun SCXW153772824 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW153772824 BCX0\">C<\/span><span class=\"NormalTextRun SCXW153772824 BCX0\">hoose Bronze layer\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW153772824 BCX0\">lakehouse<\/span><span class=\"NormalTextRun SCXW153772824 BCX0\">\u00a0item<\/span><span class=\"NormalTextRun SCXW153772824 BCX0\">.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3916039 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"3916039\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1053\" height=\"604\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture22.png\" class=\"attachment-full size-full wp-image-23384\" alt=\"Dataflow Gen2 OneLake catalog with the Bronze Layer lakehouse selected as source.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture22.png 1053w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture22-300x172.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture22-1024x587.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture22-768x441.png 768w\" sizes=\"(max-width: 1053px) 100vw, 1053px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Dataflow Gen2 OneLake catalog with the Bronze Layer lakehouse selected as source.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ef2edb0 elementor-widget elementor-widget-text-editor\" data-id=\"ef2edb0\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW113931281 BCX0\">&#8211; <span class=\"NormalTextRun SCXW3455266 BCX0\">C<\/span><span class=\"NormalTextRun SCXW3455266 BCX0\">hoose appropriate directory\/<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW3455266 BCX0\">files<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW3455266 BCX0\">.<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW3455266 BCX0\">Click<\/span><span class=\"NormalTextRun SCXW3455266 BCX0\">\u00a0on Create button.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e6c231e elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"e6c231e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1069\" height=\"501\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture23.png\" class=\"attachment-full size-full wp-image-23383\" alt=\"Dataflow Gen2 source selection of the directory and files from the Bronze layer.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture23.png 1069w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture23-300x141.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture23-1024x480.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture23-768x360.png 768w\" sizes=\"(max-width: 1069px) 100vw, 1069px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Dataflow Gen2 source selection of the directory and files from the Bronze layer.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6c4cdb0 elementor-widget elementor-widget-text-editor\" data-id=\"6c4cdb0\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW113931281 BCX0\">&#8211; <span class=\"TextRun SCXW195389980 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW195389980 BCX0\">Write commands to do transformations as per business use<\/span><span class=\"NormalTextRun SCXW195389980 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW195389980 BCX0\">case<\/span><span class=\"NormalTextRun SCXW195389980 BCX0\">\u00a0and\u00a0<\/span><span class=\"NormalTextRun SCXW195389980 BCX0\">click on \u2018<\/span><\/span><span class=\"TextRun SCXW195389980 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW195389980 BCX0\">+\u2019<\/span><\/span><span class=\"TextRun SCXW195389980 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW195389980 BCX0\">\u00a0sign besides data destination.<\/span><\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2d1f1b5 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"2d1f1b5\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1077\" height=\"405\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture24.png\" class=\"attachment-full size-full wp-image-23382\" alt=\"Dataflow Gen2 editor showing transformation steps and the add data destination button.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture24.png 1077w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture24-300x113.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture24-1024x385.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture24-768x289.png 768w\" sizes=\"(max-width: 1077px) 100vw, 1077px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Dataflow Gen2 editor showing transformation steps and the add data destination button.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-710fb16 elementor-widget elementor-widget-text-editor\" data-id=\"710fb16\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW113931281 BCX0\">&#8211; <span class=\"NormalTextRun SCXW229242749 BCX0\">Click on more and choose\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW229242749 BCX0\">Lakehouse<\/span><span class=\"NormalTextRun SCXW229242749 BCX0\">.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a94058c elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"a94058c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1064\" height=\"668\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture25.png\" class=\"attachment-full size-full wp-image-23381\" alt=\"Dataflow Gen2 data destination options with Lakehouse selected.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture25.png 1064w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture25-300x188.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture25-1024x643.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture25-768x482.png 768w\" sizes=\"(max-width: 1064px) 100vw, 1064px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Dataflow Gen2 data destination options with Lakehouse selected.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f8efa63 elementor-widget elementor-widget-text-editor\" data-id=\"f8efa63\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW113931281 BCX0\">&#8211; <span class=\"NormalTextRun SCXW178771835 BCX0\">Click on the Next button and choose<\/span><span class=\"NormalTextRun SCXW178771835 BCX0\">\u00a0existing\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW178771835 BCX0\">lakehouse<\/span><span class=\"NormalTextRun SCXW178771835 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW178771835 BCX0\">connection credentials.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c75ee71 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"c75ee71\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1081\" height=\"481\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture26.png\" class=\"attachment-full size-full wp-image-23380\" alt=\"Dataflow Gen2 destination connection setup using existing lakehouse credentials.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture26.png 1081w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture26-300x133.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture26-1024x456.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture26-768x342.png 768w\" sizes=\"(max-width: 1081px) 100vw, 1081px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Dataflow Gen2 destination connection setup using existing lakehouse credentials.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-710aa4a elementor-widget elementor-widget-text-editor\" data-id=\"710aa4a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW113931281 BCX0\">&#8211; <span class=\"TextRun SCXW179791436 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW179791436 BCX0\">Go to the\u00a0<\/span><span class=\"NormalTextRun SCXW179791436 BCX0\">appropriate workspace<\/span><span class=\"NormalTextRun SCXW179791436 BCX0\">\u00a0and\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW179791436 BCX0\">lakehouse<\/span><span class=\"NormalTextRun SCXW179791436 BCX0\"> location and choose Silver Layer as the destination.<\/span><\/span><span class=\"EOP Selected SCXW179791436 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/span><\/p><p><span class=\"TextRun SCXW220857827 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW220857827 BCX0\">&#8211; You can choose between a new<\/span><span class=\"NormalTextRun SCXW220857827 BCX0\"> table, an existing\u00a0<\/span><span class=\"NormalTextRun SCXW220857827 BCX0\">table,<\/span><span class=\"NormalTextRun SCXW220857827 BCX0\"> or a new file.<\/span><span class=\"NormalTextRun SCXW220857827 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW220857827 BCX0\">In our case, we will be choosing a new <\/span><span class=\"NormalTextRun SCXW220857827 BCX0\">table<\/span><span class=\"NormalTextRun SCXW220857827 BCX0\">.<\/span><span class=\"NormalTextRun SCXW220857827 BCX0\">\u00a0<\/span><\/span><span class=\"EOP Selected SCXW220857827 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2047fb6 elementor-widget elementor-widget-text-editor\" data-id=\"2047fb6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW113931281 BCX0\"><span class=\"TextRun SCXW152350156 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW152350156 BCX0\">&#8211; Click on Next button.<\/span><\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-758f6f0 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"758f6f0\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1051\" height=\"542\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture27.png\" class=\"attachment-full size-full wp-image-23379\" alt=\"Dataflow Gen2 destination set to the Silver Layer lakehouse with the New table option.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture27.png 1051w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture27-300x155.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture27-1024x528.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture27-768x396.png 768w\" sizes=\"(max-width: 1051px) 100vw, 1051px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Dataflow Gen2 destination set to the Silver Layer lakehouse with the New table option.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1da88b1 elementor-widget elementor-widget-text-editor\" data-id=\"1da88b1\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW113931281 BCX0\"><span class=\"TextRun SCXW152350156 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW152350156 BCX0\">&#8211; <span class=\"NormalTextRun SCXW232496159 BCX0\">It will map the columns with\u00a0<\/span><span class=\"NormalTextRun SCXW232496159 BCX0\">appropriate\u00a0<\/span><span class=\"NormalTextRun SCXW232496159 BCX0\">column<\/span><span class=\"NormalTextRun SCXW232496159 BCX0\">\u00a0data types. You can change the data types according to the use case<\/span><span class=\"NormalTextRun SCXW232496159 BCX0\">.<\/span><span class=\"NormalTextRun SCXW232496159 BCX0\">\u00a0Click on save settings.<\/span><\/span><\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f305a03 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"f305a03\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1379\" height=\"620\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture28.png\" class=\"attachment-full size-full wp-image-23378\" alt=\"Dataflow Gen2 column-to-data-type mapping screen for the destination table.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture28.png 1379w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture28-300x135.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture28-1024x460.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture28-768x345.png 768w\" sizes=\"(max-width: 1379px) 100vw, 1379px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Dataflow Gen2 column-to-data-type mapping screen for the destination table.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d4643fd elementor-widget elementor-widget-text-editor\" data-id=\"d4643fd\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW113931281 BCX0\"><span class=\"TextRun SCXW152350156 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW152350156 BCX0\">&#8211; Click the publish button<\/span><\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-91eb17c elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"91eb17c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"557\" height=\"880\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture29.png\" class=\"attachment-full size-full wp-image-23377\" alt=\"Dataflow Gen2 editor with the Publish button.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture29.png 557w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture29-190x300.png 190w\" sizes=\"(max-width: 557px) 100vw, 557px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Dataflow Gen2 editor with the Publish button.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-24e70b4 elementor-widget elementor-widget-text-editor\" data-id=\"24e70b4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW113931281 BCX0\"><span class=\"TextRun SCXW152350156 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW152350156 BCX0\">&#8211; <span class=\"TextRun SCXW49651518 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW49651518 BCX0\">We will schedule\u00a0<\/span><span class=\"NormalTextRun SCXW49651518 BCX0\">and run the\u00a0<\/span><span class=\"NormalTextRun SCXW49651518 BCX0\">Notebook\/Dataflow Gen2\u00a0<\/span><span class=\"NormalTextRun SCXW49651518 BCX0\">using<\/span><span class=\"NormalTextRun SCXW49651518 BCX0\">\u00a0data pipelines.<\/span><span class=\"NormalTextRun SCXW49651518 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW49651518 BCX0\">It is mentioned in detail under gold layer.<\/span><\/span><\/span><\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-c3b0d10 e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"c3b0d10\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-6c7c619 elementor-widget elementor-widget-heading\" data-id=\"6c7c619\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Building the Gold Layer (Curated Warehouse)<\/h3>\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6814572 elementor-widget elementor-widget-text-editor\" data-id=\"6814572\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">&#8211; This will be a warehouse. It will only contain the data that is for immediate use in the semantic model and reports. It will contain the data that is necessary for analysis purposes.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">&#8211; This will get updated either daily, weekly, or monthly, depending on the use case.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">&#8211; To create this layer, again click on new item as shown above in Bronze Layer. Search for a warehouse.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f3dcc94 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"f3dcc94\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1203\" height=\"352\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture30.png\" class=\"attachment-full size-full wp-image-23376\" alt=\"Fabric New item search showing the Warehouse option for the Gold layer.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture30.png 1203w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture30-300x88.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture30-1024x300.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture30-768x225.png 768w\" sizes=\"(max-width: 1203px) 100vw, 1203px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Fabric New item search showing the Warehouse option for the Gold layer.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-88c200d elementor-widget elementor-widget-text-editor\" data-id=\"88c200d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">&#8211; Name the warehouse and click on the Create button.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-468d1d9 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"468d1d9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"830\" height=\"431\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture31.png\" class=\"attachment-full size-full wp-image-23375\" alt=\"Create warehouse dialog with the name field and Create button.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture31.png 830w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture31-300x156.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture31-768x399.png 768w\" sizes=\"(max-width: 830px) 100vw, 830px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Create warehouse dialog with the name field and Create button.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c28c8c8 elementor-widget elementor-widget-text-editor\" data-id=\"c28c8c8\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">&#8211; <span class=\"TextRun SCXW74464406 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW74464406 BCX0\">It will\u00a0<\/span><span class=\"NormalTextRun SCXW74464406 BCX0\">contain<\/span><span class=\"NormalTextRun SCXW74464406 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW74464406 BCX0\">dbo<\/span><span class=\"NormalTextRun SCXW74464406 BCX0\"> as the default schema. Under this schema, we can create <\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW74464406 BCX0\">tables<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW74464406 BCX0\">, views<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW74464406 BCX0\">, functions,<\/span><span class=\"NormalTextRun SCXW74464406 BCX0\">\u00a0or stored procedures.<\/span><\/span><span class=\"EOP Selected SCXW74464406 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1181b39 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"1181b39\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"472\" height=\"829\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture32.png\" class=\"attachment-full size-full wp-image-23374\" alt=\"New Fabric warehouse showing the default dbo schema for tables, views, functions and stored procedures.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture32.png 472w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture32-171x300.png 171w\" sizes=\"(max-width: 472px) 100vw, 472px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">New Fabric warehouse showing the default dbo schema for tables, views, functions and stored procedures.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4835e5d elementor-widget elementor-widget-text-editor\" data-id=\"4835e5d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">&#8211; <\/span><span data-contrast=\"auto\">Since we are not doing any type of transformations on top of data stored in the Silver layer, we will directly migrate the data from the Silver layer to the Gold layer using a copy data activity in data pipelines.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">&#8211; In case one wants to do some kind of column selection or renaming or create additional columns on top of existing ones, they can do it using a Notebook\/dataflows before moving the data to the Gold layer. Steps will remain the same as mentioned under the Silver layer.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">&#8211; Now, again click on the new item as shown above in Bronze Layer. Search for pipeline.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f44cc47 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"f44cc47\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1203\" height=\"572\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture33.png\" class=\"attachment-full size-full wp-image-23373\" alt=\"Fabric New item search showing the Data pipeline option.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture33.png 1203w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture33-300x143.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture33-1024x487.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture33-768x365.png 768w\" sizes=\"(max-width: 1203px) 100vw, 1203px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Fabric New item search showing the Data pipeline option.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-63b04f3 elementor-widget elementor-widget-text-editor\" data-id=\"63b04f3\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\"><span class=\"TextRun SCXW56044424 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW56044424 BCX0\">&#8211; Name the pipeline and click on the Create button.<\/span><\/span><br \/><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1672423 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"1672423\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"716\" height=\"552\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture34.png\" class=\"attachment-full size-full wp-image-23372\" alt=\"Create data pipeline dialog with the name field and Create button.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture34.png 716w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture34-300x231.png 300w\" sizes=\"(max-width: 716px) 100vw, 716px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Create data pipeline dialog with the name field and Create button.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-935fc0d elementor-widget elementor-widget-text-editor\" data-id=\"935fc0d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\"><span class=\"TextRun SCXW56044424 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW56044424 BCX0\">&#8211; <span class=\"TextRun SCXW266165658 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW266165658 BCX0\">We will be creating two pipelines here.<\/span><span class=\"NormalTextRun SCXW266165658 BCX0\"> One to schedule the notebook\/dataflow and another to copy the data from the\u00a0<\/span><span class=\"NormalTextRun ContextualSpellingAndGrammarErrorV2Themed SCXW266165658 BCX0\">Silver<\/span><span class=\"NormalTextRun SCXW266165658 BCX0\"> layer to the Gold Layer.<\/span><\/span><span class=\"EOP Selected SCXW266165658 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/span><\/span><br \/><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-903e1b5 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"903e1b5\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1203\" height=\"432\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture35.png\" class=\"attachment-full size-full wp-image-23371\" alt=\"Empty Fabric data pipeline canvas ready for adding activities.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture35.png 1203w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture35-300x108.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture35-1024x368.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture35-768x276.png 768w\" sizes=\"(max-width: 1203px) 100vw, 1203px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Empty Fabric data pipeline canvas ready for adding activities.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-73233da e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"73233da\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-44c6d1a elementor-widget elementor-widget-heading\" data-id=\"44c6d1a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\">Bronze to silver layer pipeline<\/h4>\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-abf5425 elementor-widget elementor-widget-text-editor\" data-id=\"abf5425\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\"><span class=\"TextRun SCXW223737702 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW223737702 BCX0\">&#8211; Go to the activities tab. Click on Notebook.<\/span><\/span><span class=\"EOP Selected SCXW223737702 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><br \/><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f51d7d2 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"f51d7d2\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"849\" height=\"456\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture36.png\" class=\"attachment-full size-full wp-image-23370\" alt=\"Bronze-to-Silver pipeline with the Notebook activity added on the activities tab.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture36.png 849w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture36-300x161.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture36-768x412.png 768w\" sizes=\"(max-width: 849px) 100vw, 849px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Bronze-to-Silver pipeline with the Notebook activity added on the activities tab.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c25f05e elementor-widget elementor-widget-text-editor\" data-id=\"c25f05e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\"><span class=\"TextRun SCXW223737702 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW223737702 BCX0\">&#8211; <\/span><\/span><\/span><span data-contrast=\"auto\">Go to the Settings tab.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">&#8211; Choose the workspace where your notebook\/dataflow resides.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">&#8211; Choose the notebook\/dataflow that you want to schedule for run.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9c6be17 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"9c6be17\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1008\" height=\"972\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture37.png\" class=\"attachment-full size-full wp-image-23369\" alt=\"Notebook activity Settings tab with the workspace and notebook to schedule selected.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture37.png 1008w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture37-300x289.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture37-768x741.png 768w\" sizes=\"(max-width: 1008px) 100vw, 1008px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Notebook activity Settings tab with the workspace and notebook to schedule selected.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5e6354b elementor-widget elementor-widget-text-editor\" data-id=\"5e6354b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\"><span class=\"TextRun SCXW223737702 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW223737702 BCX0\"><span class=\"NormalTextRun SCXW118892709 BCX0\">&#8211; Go to the Run tab.<\/span><span class=\"NormalTextRun SCXW118892709 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW118892709 BCX0\">Click on &#8221; Validate &#8220;<\/span><span class=\"NormalTextRun SCXW118892709 BCX0\">\u00a0to check for any errors in your pipeline activities<\/span><span class=\"NormalTextRun SCXW118892709 BCX0\">.<\/span><span class=\"NormalTextRun SCXW118892709 BCX0\">\u00a0<\/span><br \/><\/span><\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-16959af elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"16959af\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"962\" height=\"511\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture38.png\" class=\"attachment-full size-full wp-image-23368\" alt=\"Pipeline Run tab with the Validate button to check activities for errors.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture38.png 962w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture38-300x159.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture38-768x408.png 768w\" sizes=\"(max-width: 962px) 100vw, 962px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Pipeline Run tab with the Validate button to check activities for errors.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6a45d4f elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"6a45d4f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"460\" height=\"963\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture39.png\" class=\"attachment-full size-full wp-image-23367\" alt=\"Pipeline validation output confirming no errors in the activities.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture39.png 460w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture39-143x300.png 143w\" sizes=\"(max-width: 460px) 100vw, 460px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Pipeline Run tab with the Validate button to check activities for errors.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9767183 elementor-widget elementor-widget-text-editor\" data-id=\"9767183\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\"><span class=\"TextRun SCXW223737702 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW223737702 BCX0\"><span class=\"NormalTextRun SCXW118892709 BCX0\">&#8211; <span class=\"TextRun SCXW156528984 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW156528984 BCX0\">If no errors, click on run.<\/span><\/span><\/span><br \/><\/span><\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-09850de elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"09850de\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"895\" height=\"414\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture40.png\" class=\"attachment-full size-full wp-image-23366\" alt=\"Pipeline Run tab with the Run button.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture40.png 895w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture40-300x139.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture40-768x355.png 768w\" sizes=\"(max-width: 895px) 100vw, 895px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Pipeline Run tab with the Validate button to check activities for errors.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-dc9ee0f elementor-widget elementor-widget-text-editor\" data-id=\"dc9ee0f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\"><span class=\"TextRun SCXW223737702 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW223737702 BCX0\"><span class=\"NormalTextRun SCXW118892709 BCX0\">&#8211; <span class=\"TextRun SCXW82547067 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW82547067 BCX0\">This will<\/span><span class=\"NormalTextRun SCXW82547067 BCX0\">\u00a0ask you to save and run\u00a0<\/span><span class=\"NormalTextRun SCXW82547067 BCX0\">your notebook\/dataflow for the first time.<\/span><span class=\"NormalTextRun SCXW82547067 BCX0\">\u00a0Click on save and run.<\/span><\/span><span class=\"EOP Selected SCXW82547067 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/span><br \/><\/span><\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-30e2c6c elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"30e2c6c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"700\" height=\"459\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture41.png\" class=\"attachment-full size-full wp-image-23365\" alt=\"Save and run confirmation prompt for the notebook pipeline.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture41.png 700w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture41-300x197.png 300w\" sizes=\"(max-width: 700px) 100vw, 700px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Save and run confirmation prompt for the notebook pipeline.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-de86cc3 elementor-widget elementor-widget-text-editor\" data-id=\"de86cc3\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\"><span class=\"TextRun SCXW223737702 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW223737702 BCX0\"><span class=\"NormalTextRun SCXW118892709 BCX0\">&#8211; <span class=\"NormalTextRun SCXW99382799 BCX0\">You can\u00a0<\/span><span class=\"NormalTextRun SCXW99382799 BCX0\">monitor<\/span><span class=\"NormalTextRun SCXW99382799 BCX0\">\u00a0the runs by going to View run history and then clicking Go to Monitor.<\/span><\/span><br \/><\/span><\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-17c6d11 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"17c6d11\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1203\" height=\"827\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture42.png\" class=\"attachment-full size-full wp-image-23364\" alt=\"Pipeline run history and Monitor view showing the run status.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture42.png 1203w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture42-300x206.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture42-1024x704.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture42-768x528.png 768w\" sizes=\"(max-width: 1203px) 100vw, 1203px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Pipeline run history and Monitor view showing the run status.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-c36598a e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"c36598a\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-9171312 elementor-widget elementor-widget-heading\" data-id=\"9171312\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h4 class=\"elementor-heading-title elementor-size-default\">Silver to Gold Layer pipeline<\/h4>\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8154f45 elementor-widget elementor-widget-text-editor\" data-id=\"8154f45\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\"><span class=\"TextRun SCXW223737702 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW223737702 BCX0\">&#8211; <span class=\"TextRun SCXW64417230 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW64417230 BCX0\">Go to activities tab.\u00a0<\/span><span class=\"NormalTextRun SCXW64417230 BCX0\">Click on Invoke pipeline.<\/span><\/span><span class=\"EOP Selected SCXW64417230 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/span><\/span><br \/><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-61bda51 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"61bda51\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1072\" height=\"328\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture43.png\" class=\"attachment-full size-full wp-image-23363\" alt=\"Silver-to-Gold pipeline with the Invoke pipeline activity added on the activities tab.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture43.png 1072w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture43-300x92.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture43-1024x313.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture43-768x235.png 768w\" sizes=\"(max-width: 1072px) 100vw, 1072px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Silver-to-Gold pipeline with the Invoke pipeline activity added on the activities tab.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-a222257 elementor-widget elementor-widget-text-editor\" data-id=\"a222257\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\"><span class=\"TextRun SCXW223737702 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW223737702 BCX0\">&#8211; <span class=\"TextRun SCXW25738538 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW25738538 BCX0\">Click on Settings.<\/span><span class=\"NormalTextRun SCXW25738538 BCX0\">\u00a0Type will be fabric.<\/span><\/span><span class=\"EOP Selected SCXW25738538 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/span><\/span><\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">&#8211; <span class=\"NormalTextRun SCXW253015737 BCX0\">For\u00a0<\/span><span class=\"NormalTextRun SCXW253015737 BCX0\">Connection<\/span><span class=\"NormalTextRun SCXW253015737 BCX0\">, click on Browse all.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f3dd0aa elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"f3dd0aa\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1051\" height=\"366\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture44.png\" class=\"attachment-full size-full wp-image-23362\" alt=\"Invoke pipeline Settings with type set to Fabric and the Browse all connection option.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture44.png 1051w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture44-300x104.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture44-1024x357.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture44-768x267.png 768w\" sizes=\"(max-width: 1051px) 100vw, 1051px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Invoke pipeline Settings with type set to Fabric and the Browse all connection option.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-dd31592 elementor-widget elementor-widget-text-editor\" data-id=\"dd31592\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\"><span class=\"TextRun SCXW223737702 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW223737702 BCX0\">&#8211; <span class=\"TextRun SCXW64864787 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW64864787 BCX0\">Click on Fabric data pipeline.<\/span><\/span><\/span><\/span><\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ef401ad elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"ef401ad\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"581\" height=\"370\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture45.png\" class=\"attachment-full size-full wp-image-23361\" alt=\"Connection picker with Fabric data pipeline selected.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture45.png 581w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture45-300x191.png 300w\" sizes=\"(max-width: 581px) 100vw, 581px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Connection picker with Fabric data pipeline selected.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-50dbcd5 elementor-widget elementor-widget-text-editor\" data-id=\"50dbcd5\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">&#8211; <span class=\"NormalTextRun SCXW38764864 BCX0\">Create a new connection.<\/span><span class=\"NormalTextRun SCXW38764864 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW38764864 BCX0\">Click on Connect.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-0fdbccd elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"0fdbccd\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1379\" height=\"686\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture46.png\" class=\"attachment-full size-full wp-image-23360\" alt=\"New connection dialog for the Fabric data pipeline with the Connect button.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture46.png 1379w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture46-300x149.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture46-1024x509.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture46-768x382.png 768w\" sizes=\"(max-width: 1379px) 100vw, 1379px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">New connection dialog for the Fabric data pipeline with the Connect button.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-dc066a7 elementor-widget elementor-widget-text-editor\" data-id=\"dc066a7\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">&#8211; <span class=\"NormalTextRun SCXW132366188 BCX0\">Choose the\u00a0<\/span><span class=\"NormalTextRun SCXW132366188 BCX0\">appropriate workspace<\/span><span class=\"NormalTextRun SCXW132366188 BCX0\">.<\/span><span class=\"NormalTextRun SCXW132366188 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW132366188 BCX0\">Choose\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW132366188 BCX0\">bronze_to_silver<\/span><span class=\"NormalTextRun SCXW132366188 BCX0\">\u00a0layer pipeline.<\/span><span class=\"NormalTextRun SCXW132366188 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW132366188 BCX0\">Check mark\u00a0<\/span><span class=\"NormalTextRun SCXW132366188 BCX0\">wait<\/span><span class=\"NormalTextRun SCXW132366188 BCX0\">\u00a0on completion.<\/span><span class=\"NormalTextRun SCXW132366188 BCX0\">\u00a0This will invoke bronze to silver layer pipeline.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1949c8d elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"1949c8d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1035\" height=\"473\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture47.png\" class=\"attachment-full size-full wp-image-23359\" alt=\"Invoke pipeline settings with the bronze_to_silver pipeline selected and Wait on completion checked.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture47.png 1035w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture47-300x137.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture47-1024x468.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture47-768x351.png 768w\" sizes=\"(max-width: 1035px) 100vw, 1035px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Invoke pipeline settings with the bronze_to_silver pipeline selected and Wait on completion checked.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c1a7c6c elementor-widget elementor-widget-text-editor\" data-id=\"c1a7c6c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">&#8211; <span class=\"NormalTextRun SCXW102619655 BCX0\">Now,\u00a0<\/span><span class=\"NormalTextRun SCXW102619655 BCX0\">Click on copy data.<\/span><span class=\"NormalTextRun SCXW102619655 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW102619655 BCX0\">Click on Add copy data activity.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c4e6ada elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"c4e6ada\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"921\" height=\"377\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture48.png\" class=\"attachment-full size-full wp-image-23358\" alt=\"Pipeline canvas with the Copy data activity being added.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture48.png 921w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture48-300x123.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture48-768x314.png 768w\" sizes=\"(max-width: 921px) 100vw, 921px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Pipeline canvas with the Copy data activity being added.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ac297db elementor-widget elementor-widget-text-editor\" data-id=\"ac297db\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">&#8211; <span class=\"TextRun SCXW186397358 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW186397358 BCX0\">Click on Source.<\/span><\/span><\/span><\/p><p>&#8211; <span class=\"TextRun SCXW75210269 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW75210269 BCX0\">Connection will be Lakehouse<\/span><span class=\"NormalTextRun SCXW75210269 BCX0\"> and Lakehouse will be Silver Layer L<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW75210269 BCX0\">akehouse<\/span><span class=\"NormalTextRun SCXW75210269 BCX0\">.<\/span><\/span><span class=\"EOP Selected SCXW75210269 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/p><p>&#8211; <span class=\"NormalTextRun SCXW251984883 BCX0\">We are saving our data as tables in silver layer. So,\u00a0<\/span><span class=\"NormalTextRun SCXW251984883 BCX0\">our root folder will be Tables<\/span><span class=\"NormalTextRun SCXW251984883 BCX0\">,<\/span><span class=\"NormalTextRun SCXW251984883 BCX0\">\u00a0and\u00a0<\/span><span class=\"NormalTextRun SCXW251984883 BCX0\">th<\/span><span class=\"NormalTextRun SCXW251984883 BCX0\">e query will be Table.<\/span><\/p><p>&#8211; <span class=\"TextRun SCXW31596939 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW31596939 BCX0\">We will choose our\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW31596939 BCX0\">praxedo<\/span><span class=\"NormalTextRun SCXW31596939 BCX0\">\u00a0table from the drop<\/span><span class=\"NormalTextRun SCXW31596939 BCX0\">&#8211;<\/span><span class=\"NormalTextRun SCXW31596939 BCX0\">down.<\/span><\/span><span class=\"EOP Selected SCXW31596939 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7224523 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"7224523\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1014\" height=\"873\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture49.png\" class=\"attachment-full size-full wp-image-23357\" alt=\"Copy data Source settings with the Silver Layer lakehouse and the source table selected from Tables.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture49.png 1014w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture49-300x258.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture49-768x661.png 768w\" sizes=\"(max-width: 1014px) 100vw, 1014px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Pipeline canvas with the Copy data activity being added.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-29c99e6 elementor-widget elementor-widget-text-editor\" data-id=\"29c99e6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">&#8211; <span class=\"TextRun SCXW97158514 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW97158514 BCX0\">Now, choose destination.<\/span><\/span><\/span><\/p><p>&#8211; <span class=\"NormalTextRun SCXW233399882 BCX0\">Connection will be\u00a0<\/span><span class=\"NormalTextRun ContextualSpellingAndGrammarErrorV2Themed SCXW233399882 BCX0\">Gold<\/span><span class=\"NormalTextRun SCXW233399882 BCX0\">\u00a0layer. Table\u00a0<\/span><span class=\"NormalTextRun SCXW233399882 BCX0\">option<\/span><span class=\"NormalTextRun SCXW233399882 BCX0\">\u00a0will be Auto\u00a0<\/span><span class=\"NormalTextRun ContextualSpellingAndGrammarErrorV2Themed SCXW233399882 BCX0\">create<\/span><span class=\"NormalTextRun SCXW233399882 BCX0\">\u00a0table.<\/span><\/p><p>&#8211; <span class=\"NormalTextRun SCXW196564472 BCX0\">Provide the schema name and the table name.<\/span><span class=\"NormalTextRun SCXW196564472 BCX0\">\u00a0Write behaviour will be insert.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f4a6e77 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"f4a6e77\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1014\" height=\"335\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture50.png\" class=\"attachment-full size-full wp-image-23356\" alt=\"Copy data Destination settings with the Gold warehouse, Auto create table option and Insert write behaviour.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture50.png 1014w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture50-300x99.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture50-768x254.png 768w\" sizes=\"(max-width: 1014px) 100vw, 1014px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Copy data Destination settings with the Gold warehouse, Auto create table option and Insert write behaviour.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7c4e79d elementor-widget elementor-widget-text-editor\" data-id=\"7c4e79d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">&#8211; <span class=\"NormalTextRun SCXW27225823 BCX0\">Click on Mapping. Click on import schemas.<\/span><span class=\"NormalTextRun SCXW27225823 BCX0\">\u00a0This will map the source and destination table columns automatically.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e1fcebe elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"e1fcebe\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1379\" height=\"501\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture51.png\" class=\"attachment-full size-full wp-image-23355\" alt=\"Copy data Mapping tab after importing schemas to map source and destination columns.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture51.png 1379w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture51-300x109.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture51-1024x372.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture51-768x279.png 768w\" sizes=\"(max-width: 1379px) 100vw, 1379px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Copy data Mapping tab after importing schemas to map source and destination columns.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-298c1c8 elementor-widget elementor-widget-text-editor\" data-id=\"298c1c8\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">&#8211; <span class=\"NormalTextRun SCXW47262878 BCX0\">Ensure that under Settings tab,\u00a0<span class=\"NormalTextRun ContextualSpellingAndGrammarErrorV2Themed SCXW47262878 BCX0\">Enable<\/span>\u00a0staging is ON\u00a0and data store type is Workspace.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f1fe5d9 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"f1fe5d9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"994\" height=\"640\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture52.png\" class=\"attachment-full size-full wp-image-23354\" alt=\"Copy data Settings tab with Enable staging on and data store type set to Workspace.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture52.png 994w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture52-300x193.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture52-768x494.png 768w\" sizes=\"(max-width: 994px) 100vw, 994px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Copy data Settings tab with Enable staging on and data store type set to Workspace.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7cb9267 elementor-widget elementor-widget-text-editor\" data-id=\"7cb9267\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">&#8211; <span class=\"NormalTextRun SCXW78793106 BCX0\">Now create a<\/span><span class=\"NormalTextRun SCXW78793106 BCX0\">n arrow\u00a0<\/span><span class=\"NormalTextRun SCXW78793106 BCX0\">from Invoke pipeline to copy data activity<\/span><span class=\"NormalTextRun SCXW78793106 BCX0\">. These activities will then run in sequence.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1886518 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"1886518\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1051\" height=\"352\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture53.png\" class=\"attachment-full size-full wp-image-23353\" alt=\"Pipeline canvas with an arrow connecting the Invoke pipeline activity to the Copy data activity in sequence.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture53.png 1051w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture53-300x100.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture53-1024x343.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture53-768x257.png 768w\" sizes=\"(max-width: 1051px) 100vw, 1051px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Pipeline canvas with an arrow connecting the Invoke pipeline activity to the Copy data activity in sequence.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-bad21c2 elementor-widget elementor-widget-text-editor\" data-id=\"bad21c2\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">&#8211; <span class=\"NormalTextRun SCXW251236220 BCX0\">Go to Run tab.<\/span><span class=\"NormalTextRun SCXW251236220 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW251236220 BCX0\">Click on\u00a0<\/span><span class=\"NormalTextRun SCXW251236220 BCX0\">validate<\/span><span class=\"NormalTextRun SCXW251236220 BCX0\">\u00a0to check for any errors in your pipeline activities<\/span><span class=\"NormalTextRun SCXW251236220 BCX0\"> as shown <\/span><span class=\"NormalTextRun SCXW251236220 BCX0\">in\u00a0<\/span><span class=\"NormalTextRun SCXW251236220 BCX0\">previous<\/span><span class=\"NormalTextRun SCXW251236220 BCX0\">\u00a0steps<\/span><span class=\"NormalTextRun SCXW251236220 BCX0\">.<\/span><\/span><\/p><p>&#8211; <span class=\"TextRun SCXW102509248 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW102509248 BCX0\">If no errors, click on run. This will<\/span><span class=\"NormalTextRun SCXW102509248 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW102509248 BCX0\">ask you to save and\u00a0<\/span><span class=\"NormalTextRun SCXW102509248 BCX0\">run your pipeline for the first time.<\/span><span class=\"NormalTextRun SCXW102509248 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW102509248 BCX0\">Click on save and run.\u00a0<\/span><span class=\"NormalTextRun SCXW102509248 BCX0\">Bronze to Silver layer pipeline will also run<\/span><span class=\"NormalTextRun SCXW102509248 BCX0\">\u00a0and that will simply overwrite the data from Bronze to Silver layer.<\/span><\/span><span class=\"EOP Selected SCXW102509248 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/p><p>&#8211; <span class=\"TextRun SCXW218616932 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW218616932 BCX0\">Once the run is successful, go back to Activities tab and click on copy data activity.<\/span><\/span><\/p><p>&#8211; <span class=\"TextRun SCXW68167748 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW68167748 BCX0\">Now, change the table\u00a0<\/span><span class=\"NormalTextRun SCXW68167748 BCX0\">option<\/span><span class=\"NormalTextRun SCXW68167748 BCX0\">\u00a0from Auto create table to Use existing and choose the\u00a0<\/span><span class=\"NormalTextRun SCXW68167748 BCX0\">appropriate destination<\/span><span class=\"NormalTextRun SCXW68167748 BCX0\">\u00a0table\u00a0<\/span><span class=\"NormalTextRun SCXW68167748 BCX0\">from the drop down.<\/span><\/span><span class=\"EOP Selected SCXW68167748 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/p><p>&#8211; <span class=\"NormalTextRun SCXW189926258 BCX0\">Also, change the write behaviour from insert to\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW189926258 BCX0\">upsert<\/span><span class=\"NormalTextRun SCXW189926258 BCX0\">. It will ask for\u00a0<\/span><span class=\"NormalTextRun SCXW189926258 BCX0\">Key columns which will help it to\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW189926258 BCX0\">analyze<\/span><span class=\"NormalTextRun SCXW189926258 BCX0\">\u00a0<\/span><span class=\"NormalTextRun SCXW189926258 BCX0\">which data is to insert and which data to update accordingly.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3941e69 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"3941e69\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1204\" height=\"532\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture54.png\" class=\"attachment-full size-full wp-image-23352\" alt=\"Copy data Destination updated to Use existing table with Upsert write behaviour and key columns selected.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture54.png 1204w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture54-300x133.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture54-1024x452.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture54-768x339.png 768w\" sizes=\"(max-width: 1204px) 100vw, 1204px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Copy data Destination updated to Use existing table with Upsert write behaviour and key columns selected.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f65ab35 elementor-widget elementor-widget-text-editor\" data-id=\"f65ab35\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p>&#8211; <span class=\"TextRun SCXW71629353 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW71629353 BCX0\">Go back to Home tab.<\/span><\/span><\/p><p>&#8211; <span class=\"TextRun SCXW86729967 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW86729967 BCX0\">Click on Save button.<\/span><\/span><\/p><p>&#8211; <span class=\"TextRun SCXW176912765 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW176912765 BCX0\">This will save your pipeline.<\/span><\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-eb69636 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"eb69636\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"971\" height=\"402\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture55.png\" class=\"attachment-full size-full wp-image-23351\" alt=\"Pipeline Home tab with the Save button.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture55.png 971w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture55-300x124.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture55-768x318.png 768w\" sizes=\"(max-width: 971px) 100vw, 971px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Pipeline Home tab with the Save button.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3ff076a elementor-widget elementor-widget-text-editor\" data-id=\"3ff076a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p>&#8211; <span class=\"TextRun SCXW186518434 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW186518434 BCX0\">Click<\/span><span class=\"NormalTextRun SCXW186518434 BCX0\">\u00a0on add schedule.\u00a0<\/span><span class=\"NormalTextRun SCXW186518434 BCX0\">Based on your use case, you can schedule it by minute, hourly, daily,\u00a0<\/span><span class=\"NormalTextRun SCXW186518434 BCX0\">weekly<\/span><span class=\"NormalTextRun SCXW186518434 BCX0\">\u00a0or monthly.<\/span><\/span><span class=\"EOP Selected SCXW186518434 BCX0\" data-ccp-props=\"{}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ece0c70 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"ece0c70\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1204\" height=\"1110\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture57.png\" class=\"attachment-full size-full wp-image-23349\" alt=\"Pipeline schedule configuration showing frequency options from minute to monthly.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture57.png 1204w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture57-300x277.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture57-1024x944.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture57-768x708.png 768w\" sizes=\"(max-width: 1204px) 100vw, 1204px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Pipeline schedule configuration showing frequency options from minute to monthly.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-1f5b17e elementor-widget elementor-widget-text-editor\" data-id=\"1f5b17e\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p>&#8211; <span class=\"NormalTextRun SCXW90542637 BCX0\">You can\u00a0<\/span><span class=\"NormalTextRun SCXW90542637 BCX0\">add the frequency. In our case, we need to run it on the daily basis.<\/span><span class=\"NormalTextRun SCXW90542637 BCX0\">\u00a0You can add\u00a0<\/span><span class=\"NormalTextRun SCXW90542637 BCX0\">attributes such as the\u00a0<\/span><span class=\"NormalTextRun SCXW90542637 BCX0\">time at which you want to\u00a0<\/span><span class=\"NormalTextRun SCXW90542637 BCX0\">schedule the pipeline<\/span><span class=\"NormalTextRun SCXW90542637 BCX0\">,\u00a0<\/span><span class=\"NormalTextRun SCXW90542637 BCX0\">start date, end\u00a0<\/span><span class=\"NormalTextRun SCXW90542637 BCX0\">date<\/span><span class=\"NormalTextRun SCXW90542637 BCX0\">\u00a0and time zone.<\/span><span class=\"NormalTextRun SCXW90542637 BCX0\">\u00a0After this, click on save button.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5a1fe2f elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"5a1fe2f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1015\" height=\"797\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture58.png\" class=\"attachment-full size-full wp-image-23348\" alt=\"Schedule settings set to run daily, with time, start date, end date and time zone fields.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture58.png 1015w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture58-300x236.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture58-768x603.png 768w\" sizes=\"(max-width: 1015px) 100vw, 1015px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Schedule settings set to run daily, with time, start date, end date and time zone fields.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-43bf8bb elementor-widget elementor-widget-text-editor\" data-id=\"43bf8bb\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p>&#8211; <span class=\"NormalTextRun SCXW103982056 BCX0\">Also, you can add the\u00a0<\/span><span class=\"NormalTextRun SCXW103982056 BCX0\">email addresses at which the notification will be sent if the pipeline fails on a particular day.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-b54cbc8 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"b54cbc8\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1051\" height=\"319\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture59.png\" class=\"attachment-full size-full wp-image-23347\" alt=\"Pipeline failure notification settings with email recipient fields.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture59.png 1051w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture59-300x91.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture59-1024x311.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture59-768x233.png 768w\" sizes=\"(max-width: 1051px) 100vw, 1051px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Pipeline failure notification settings with email recipient fields.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-dfe4e9a e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"dfe4e9a\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-0dab710 elementor-widget elementor-widget-heading\" data-id=\"0dab710\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">OVERALL ARCHITECTURE <\/h2>\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-4e74c61 elementor-widget elementor-widget-text-editor\" data-id=\"4e74c61\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p>&#8211; <span class=\"NormalTextRun SCXW123488592 BCX0\">We\u00a0<\/span><span class=\"NormalTextRun SCXW123488592 BCX0\">have following\u00a0<\/span><span class=\"NormalTextRun SCXW123488592 BCX0\">2<\/span><span class=\"NormalTextRun SCXW123488592 BCX0\">\u00a0folders. Current architecture is under development folder.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-39ae0a4 elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"39ae0a4\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1204\" height=\"313\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture60.png\" class=\"attachment-full size-full wp-image-23346\" alt=\"BI PROD \u2013 Group Technical Services Fabric workspace showing the Development and Production folders.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture60.png 1204w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture60-300x78.png 300w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture60-1024x266.png 1024w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture60-768x200.png 768w\" sizes=\"(max-width: 1204px) 100vw, 1204px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">BI PROD \u2013 Group Technical Services Fabric workspace showing the Development and Production folders.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-61fcbe2 elementor-widget elementor-widget-text-editor\" data-id=\"61fcbe2\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t\t\t\t\t\t<p>&#8211; <span data-contrast=\"auto\">Under development folder, we will have\u00a0following\u00a0architecture.\u00a0Our layers will be kept here.\u00a0All the other items will be kept under respective folders.<\/span><span data-ccp-props=\"{}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-81e14ce elementor-widget__width-initial elementor-widget elementor-widget-image\" data-id=\"81e14ce\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"1203\" height=\"1274\" src=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture61.png\" class=\"attachment-full size-full wp-image-23345\" alt=\"Development folder showing the Bronze and Silver lakehouses, Gold warehouse, and Notebooks, Pipelines and Reports subfolders.\" srcset=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture61.png 1203w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture61-283x300.png 283w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture61-967x1024.png 967w, https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/Picture61-768x813.png 768w\" sizes=\"(max-width: 1203px) 100vw, 1203px\" \/>\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Development folder showing the Bronze and Silver lakehouses, Gold warehouse, and Notebooks, Pipelines and Reports subfolders.<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t<div data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-element elementor-element-45c2ab1 e-con-full e-ecs-flex e-flex e-con e-child\" data-id=\"45c2ab1\" data-element_type=\"container\" data-e-type=\"container\" data-settings=\"{&quot;ecs_container_type&quot;:&quot;flex&quot;}\">\n\t\t\t\t<div class=\"elementor-element elementor-element-a842c41 elementor-widget elementor-widget-heading\" data-id=\"a842c41\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Frequently Asked Questions (FAQ's)<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5922589 elementor-widget elementor-widget-eael-adv-accordion\" data-id=\"5922589\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"eael-adv-accordion.default\">\n\t\t\t\t\t            <div class=\"eael-adv-accordion\" id=\"eael-adv-accordion-5922589\" data-scroll-on-click=\"no\" data-scroll-speed=\"300\" data-accordion-id=\"5922589\" data-accordion-type=\"accordion\" data-toogle-speed=\"300\">\n            <div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"is-it-good-to-get-data-directly-through-dataflow-gen-2-or-is-it-better-to-load-the-data-first-to-onelake-using-shortcuts-and-then-reference-that-data-to-dataflow-gen-2-\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"1\" aria-controls=\"elementor-tab-content-9341\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">Is it good to get data directly through dataflow Gen 2 or is it better to load the data first to Onelake using shortcuts and then reference that data to dataflow Gen 2? <\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-9341\" class=\"eael-accordion-content clearfix\" data-tab=\"1\" aria-labelledby=\"is-it-good-to-get-data-directly-through-dataflow-gen-2-or-is-it-better-to-load-the-data-first-to-onelake-using-shortcuts-and-then-reference-that-data-to-dataflow-gen-2-\"><p><span>Dataflow Gen2 pulls data directly from the source:<\/span><span>\u00a0<\/span><\/p><p><span>Dataflow \u2192 Source system \u2192 Transform \u2192\u00a0OneLake<\/span><span>\u00a0<\/span><\/p><p><span>Benefits:<\/span><span>\u00a0<\/span><\/p><ul><li><span>Simple end\u2011to\u2011end pipeline<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Less infrastructure to manage<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Good for operational\/transactional sources that change\u00a0frequently<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Good if you need to transform the data while loading<\/span><span>\u00a0<\/span><\/li><\/ul><p><span>Limitations:<\/span><span>\u00a0<\/span><\/p><ul><li><span>Repeatedly hitting the source system can cause:<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Load on transactional systems<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Throttling (e.g., Dataverse, SharePoint)<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Higher latency<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>No centralized \u201csource\u2011of\u2011truth\u201d layer in\u00a0OneLake<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Harder for other teams\/tools to reuse the raw data<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>You\u00a0can&#8217;t\u00a0reuse the same ingested dataset efficiently across multiple Dataflows<\/span><span>\u00a0<\/span><\/li><\/ul><\/div>\n\t\t\t\t\t<\/div><div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"load-to-onelake-first-via-shortcuts-or-ingestion-then-let-dataflow-gen2-read-from-onelake\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"2\" aria-controls=\"elementor-tab-content-9342\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">Load to OneLake first (via shortcuts or ingestion), then let Dataflow Gen2 read from OneLake:<\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-9342\" class=\"eael-accordion-content clearfix\" data-tab=\"2\" aria-labelledby=\"load-to-onelake-first-via-shortcuts-or-ingestion-then-let-dataflow-gen2-read-from-onelake\"><p><span>Source \u2192\u00a0OneLake\u00a0shortcut\/landing \u2192 Dataflow \u2192 curated data<\/span><span>\u00a0<\/span><\/p><p><span>Benefits:<\/span><span>\u00a0<\/span><\/p><ul><li><span>Eliminates\u00a0repeated source-system load<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Dataflow Gen2 reads local\u00a0OneLake\u00a0files,\u00a0no external load.<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>You get a reusable bronze layer<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Other teams\/tools can use the same raw data<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Best for a\u00a0lakehouse\u00a0architecture<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Dataflow Gen2 fits nicely in Silver\/Gold.<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Shortcuts make external data feel local<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>You avoid copying data but still get:<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Local\u00a0OneLake\u00a0performance<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Delta format compatibility<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Ability to combine data from ADLS, S3, Dataverse, other workspaces<\/span><span>\u00a0<\/span><\/li><li><span>Lower latency + higher stability<\/span><span>\u00a0<\/span><\/li><li><span>Reading from\u00a0OneLake\u00a0is much faster than calling external APIs repeatedly.<\/span><\/li><\/ul><p><span>Limitations:<\/span><span>\u00a0<\/span><\/p><ul><li><span>Requires\u00a0an initial\u00a0landing step (shortcut or separate ingestion)<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Slightly more architecture decisions<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Data freshness may depend on how often the source updates<\/span><span>\u00a0<\/span><\/li><\/ul><p><span>Hence second option will help in scalability, reusability, will be engine\u2011agnostic, useful for enterprise\u2011grade architecture.<\/span><span>\u00a0<\/span><\/p><\/div>\n\t\t\t\t\t<\/div><div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"when-to-use-medallion-architecture\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"3\" aria-controls=\"elementor-tab-content-9343\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">When to use medallion architecture?<\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-9343\" class=\"eael-accordion-content clearfix\" data-tab=\"3\" aria-labelledby=\"when-to-use-medallion-architecture\"><p><span>You can use medallion architecture in following scenarios:<\/span><span>\u00a0<\/span><\/p><ul><li><span>When You Need a Reusable Raw Data Layer (Bronze)<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>When Transformations Happen in Stages (Silver \u2192 Gold)<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>When You Need Separation of Storage + Compute Layers<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>When You Want Enterprise-Grade Architecture<\/span><span>\u00a0<\/span><\/li><\/ul><p><span>\u00a0<\/span><\/p><\/div>\n\t\t\t\t\t<\/div><div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"when-not-to-use-medallion-architecture\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"4\" aria-controls=\"elementor-tab-content-9344\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">When not to use medallion architecture?<\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-9344\" class=\"eael-accordion-content clearfix\" data-tab=\"4\" aria-labelledby=\"when-not-to-use-medallion-architecture\"><p><span>You cannot use medallion architecture in following scenarios:<\/span><span>\u00a0<\/span><\/p><ul><li><span>When Data Volumes Are Very Small or Simple<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>When You Need\u00a0Real<\/span>\u2011<span>Time\u00a0or Near\u00a0Real<\/span>\u2011<span>Time\u00a0Streaming<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>When Source System Direct Pull Is Simple and Does Not Cause Load<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>When There Is No Need for Reusability Across Teams<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>When You\u00a0Don\u2019t\u00a0Need Warehousing or Star Schema Models<\/span><span>\u00a0<\/span><\/li><\/ul><p><span>\u00a0<\/span><\/p><\/div>\n\t\t\t\t\t<\/div><div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"where-shortcuts-will-work\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"5\" aria-controls=\"elementor-tab-content-9345\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">Where shortcuts will work?<\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-9345\" class=\"eael-accordion-content clearfix\" data-tab=\"5\" aria-labelledby=\"where-shortcuts-will-work\"><p><span class=\"TextRun SCXW19292234 BCX0\"><span class=\"NormalTextRun SpellingErrorV2Themed SCXW19292234 BCX0\">OneLake<\/span><span class=\"NormalTextRun SCXW19292234 BCX0\">\u00a0shortcuts only work with file-based object storage (S3\/GCS\/ADLS-style endpoints).<\/span><\/span><span class=\"EOP SCXW19292234 BCX0\">\u00a0<\/span><\/p><\/div>\n\t\t\t\t\t<\/div><div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"where-shortcuts-will-wont-work\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"6\" aria-controls=\"elementor-tab-content-9346\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">Where shortcuts will won\u2019t work?<\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-9346\" class=\"eael-accordion-content clearfix\" data-tab=\"6\" aria-labelledby=\"where-shortcuts-will-wont-work\"><p><span class=\"TextRun SCXW202691790 BCX0\"><span class=\"NormalTextRun SCXW202691790 BCX0\">SQL Server is not object storage, so it cannot be mapped as a shortcut.<\/span><\/span><span class=\"EOP SCXW202691790 BCX0\">\u00a0<\/span><\/p><\/div>\n\t\t\t\t\t<\/div><div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"what-are-the-alternatives-if-data-is-stored-in-sql-server-\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"7\" aria-controls=\"elementor-tab-content-9347\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">What are the alternatives if data is stored in SQL server? <\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-9347\" class=\"eael-accordion-content clearfix\" data-tab=\"7\" aria-labelledby=\"what-are-the-alternatives-if-data-is-stored-in-sql-server-\"><p><b><span>Alternatives:<\/span><\/b><span>\u00a0<\/span><\/p><ul><li><span>Use Mirroring for SQL Server (Recommended) &#8211; It uses CDC and hence tables which fulfil CDC criteria will be copied.<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Use Dataflow Gen2 directly &#8211; Load time will increase<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Use Data Pipelines to load data to\u00a0lakehouse<\/span><span>\u00a0<\/span><\/li><\/ul><\/div>\n\t\t\t\t\t<\/div><div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"what-is-mirroring\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"8\" aria-controls=\"elementor-tab-content-9348\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">What is mirroring?<\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-9348\" class=\"eael-accordion-content clearfix\" data-tab=\"8\" aria-labelledby=\"what-is-mirroring\"><p><span class=\"TextRun SCXW96674639 BCX0\"><span class=\"NormalTextRun SCXW96674639 BCX0\">Mirroring creates a real-time or near\u2013real-time replica of your SQL Server tables into\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW96674639 BCX0\">OneLake<\/span><span class=\"NormalTextRun SCXW96674639 BCX0\">\u00a0using CDC.<\/span><\/span><span class=\"EOP SCXW96674639 BCX0\">\u00a0<\/span><\/p><\/div>\n\t\t\t\t\t<\/div><div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"where-can-we-use-mirroring\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"9\" aria-controls=\"elementor-tab-content-9349\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">Where can we use mirroring?<\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-9349\" class=\"eael-accordion-content clearfix\" data-tab=\"9\" aria-labelledby=\"where-can-we-use-mirroring\"><p><span>We should use mirroring:<\/span><span>\u00a0<\/span><\/p><ul><li><span>When the Source System Is SQL Server<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>When You Need Near-Real-Time Sync<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>When You Want to Avoid Full Loads<\/span><span>\u00a0<\/span><\/li><\/ul><\/div>\n\t\t\t\t\t<\/div><div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"where-cant-we-use-mirroring\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"10\" aria-controls=\"elementor-tab-content-93410\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">Where can\u2019t we use mirroring?<\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-93410\" class=\"eael-accordion-content clearfix\" data-tab=\"10\" aria-labelledby=\"where-cant-we-use-mirroring\"><p><span>Tables excluded from mirroring:<\/span><span>\u00a0<\/span><\/p><ul><li><span>No primary key &#8211; CDC cannot track tables without PK<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Unsupported schema &#8211; FILESTREAM, in\u2011memory OLTP, sparse, computed columns<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Unsupported datatypes &#8211; CLR types, geography, geometry,\u00a0sql_variant<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Temporal\/history tables &#8211; Not supported by CDC-based mirroring<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Missing permissions &#8211; Fabric login lacks SELECT\/ALTER ANY EXTERNAL MIRROR<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Already in replication &#8211; CDC conflict blocks mirroring<\/span><span>\u00a0<\/span><\/li><\/ul><ul><li><span>System tables &#8211; Fabric mirrors user tables only<\/span><span>\u00a0<\/span><\/li><\/ul><\/div>\n\t\t\t\t\t<\/div><div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"what-is-data-masking\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"11\" aria-controls=\"elementor-tab-content-93411\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">What is data masking?<\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-93411\" class=\"eael-accordion-content clearfix\" data-tab=\"11\" aria-labelledby=\"what-is-data-masking\"><p><b><span>Data masking in Microsoft Fabric<\/span><\/b><span>\u00a0is a security feature that protects sensitive data by obscuring it for unauthorized or\u00a0low<\/span>\u2011<span>privileged\u00a0users, while still enabling analytics, reporting, and development work.<\/span><span>\u00a0<\/span><\/p><p><span>Microsoft Fabric supports\u00a0<\/span><b><span>Dynamic Data Masking (DDM)<\/span><\/b><span>\u00a0natively in\u00a0<\/span><b><span>Fabric Data Warehouse<\/span><\/b><span>\u00a0and the\u00a0<\/span><b><span>SQL analytics endpoint<\/span><\/b><span>.\u00a0In our case, we are using\u00a0<\/span><b><span>Default mask<\/span><\/b><span>\u00a0(e.g., strings become XXXX, numeric become 0, dates become 1900-01-01).<\/span><span>\u00a0<\/span><\/p><\/div>\n\t\t\t\t\t<\/div><div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"who-can-view-masked-data\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"12\" aria-controls=\"elementor-tab-content-93412\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">Who can view masked data?<\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-93412\" class=\"eael-accordion-content clearfix\" data-tab=\"12\" aria-labelledby=\"who-can-view-masked-data\"><p><span class=\"TextRun SCXW60341435 BCX0\"><span class=\"NormalTextRun SCXW60341435 BCX0\">Roles such as Admin, Member, or Contributor have CONTROL permission on the database by design and can view\u00a0<\/span><span class=\"NormalTextRun SCXW60341435 BCX0\">un<\/span><span class=\"NormalTextRun SCXW60341435 BCX0\">masked data by default.<\/span><\/span><span class=\"EOP SCXW60341435 BCX0\">\u00a0<\/span><\/p><\/div>\n\t\t\t\t\t<\/div><div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"who-cannot-view-the-masked-data-\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"13\" aria-controls=\"elementor-tab-content-93413\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">Who cannot view the masked data? <\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-93413\" class=\"eael-accordion-content clearfix\" data-tab=\"13\" aria-labelledby=\"who-cannot-view-the-masked-data-\"><p><span class=\"TextRun SCXW251941225 BCX0\"><span class=\"NormalTextRun SCXW251941225 BCX0\">Users without the Administrator, Member, or Contributor rights on the workspace, and without elevated permissions on the Warehouse, will see masked data.<\/span><\/span><span class=\"EOP SCXW251941225 BCX0\">\u00a0<\/span><\/p><\/div>\n\t\t\t\t\t<\/div><div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"what-is-the-query-to-mask-the-data-in-a-particular-column\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"14\" aria-controls=\"elementor-tab-content-93414\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">What is the query to mask the data in a particular column?<\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-93414\" class=\"eael-accordion-content clearfix\" data-tab=\"14\" aria-labelledby=\"what-is-the-query-to-mask-the-data-in-a-particular-column\"><p><span>&#8211; -Applying mask on a particular column &#8211; &#8211;<\/span><span>\u00a0<\/span><\/p><p><span>ALTER TABLE\u00a0dbo.pxo_validated_tickets<\/span><span>\u00a0<\/span><\/p><p><span>ALTER COLUMN Id ADD MASKED WITH (FUNCTION = &#8216;default()&#8217;);<\/span><span>\u00a0<\/span><\/p><p><span>GO<\/span><span>\u00a0<\/span><\/p><p><span>&#8211; &#8211; Granting\/Revoking permissions for a desired user on masked data column &#8211; &#8211;<\/span><span>\u00a0<\/span><\/p><p><span>GRANT UNMASK ON\u00a0dbo.pxo_validated_tickets\u00a0TO\u00a0sabhye.singhal@synlab.com;<\/span><span>\u00a0<\/span><\/p><p><span>REVOKE UNMASK ON\u00a0dbo.pxo_validated_tickets\u00a0TO\u00a0sabhye.singhal@synlab.com;<\/span><span>\u00a0<\/span><\/p><p><span>&#8211; &#8211; Revoke permissions on a specific group &#8211; &#8211;<\/span><span>\u00a0<\/span><\/p><p><span>ALTER ROLE &lt;YourRoleName&gt; REVOKE\u00a0UNMASK;<\/span><\/p><\/div>\n\t\t\t\t\t<\/div><div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"what-is-row-level-security\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"15\" aria-controls=\"elementor-tab-content-93415\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">What is row-level security?<\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-93415\" class=\"eael-accordion-content clearfix\" data-tab=\"15\" aria-labelledby=\"what-is-row-level-security\"><p><b><span>Row-Level Security (RLS)<\/span><\/b><span>\u00a0in Microsoft Fabric allows you to restrict access to specific rows in a table based on\u00a0<\/span><b><span>who the user is<\/span><\/b><span>\u00a0or\u00a0<\/span><b><span>what role\/group they belong to<\/span><\/b><span>. This ensures each user sees\u00a0<\/span><b><span>only the data\u00a0they\u2019re\u00a0permitted\u00a0to see<\/span><\/b><span>, even when everyone queries the same table.<\/span><span>\u00a0<\/span><\/p><p><span>According to Microsoft Learn, RLS applies to both the\u00a0<\/span><b><span>Fabric Warehouse<\/span><\/b><span>\u00a0and the\u00a0<\/span><b><span>SQL analytics endpoint<\/span><\/b><span>.<\/span><span>\u00a0<\/span><\/p><p><span>RLS enables precise control by filtering rows at query time:<\/span><span>\u00a0<\/span><\/p><ul><li><span>Workers only see rows for\u00a0<\/span><b><span>their department<\/span><\/b><span>\u00a0<\/span><\/li><\/ul><ul><li><span>Customers only see\u00a0<\/span><b><span>their organization\u2019s data<\/span><\/b><span>\u00a0in a multitenant setup\u00a0<\/span><span>\u00a0<\/span><\/li><\/ul><p><span>RLS is enforced\u00a0<\/span><b><span>inside the database tier<\/span><\/b><span>, not in apps like Power BI, ensuring consistent security across all tools and reducing the risk of bypass.<\/span><\/p><\/div>\n\t\t\t\t\t<\/div><div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"in-which-scenarios-rls-wont-work\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"16\" aria-controls=\"elementor-tab-content-93416\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">In which scenarios RLS won't work?<\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-93416\" class=\"eael-accordion-content clearfix\" data-tab=\"16\" aria-labelledby=\"in-which-scenarios-rls-wont-work\"><p><span class=\"TextRun SCXW57888485 BCX0\"><span class=\"NormalTextRun SCXW57888485 BCX0\">Users with\u00a0<\/span><\/span><span class=\"TextRun SCXW57888485 BCX0\"><span class=\"NormalTextRun SCXW57888485 BCX0\">Admin, Member, or Contributor permissions<\/span><\/span><span class=\"TextRun SCXW57888485 BCX0\"><span class=\"NormalTextRun SCXW57888485 BCX0\">\u00a0at the workspace level inherently have\u00a0<\/span><span class=\"NormalTextRun SpellingErrorV2Themed SCXW57888485 BCX0\">high<\/span><\/span><span class=\"NoBreakHyphenBlob BlobObject DragDrop SpellingErrorV2Themed SCXW57888485 BCX0\">\u2011<\/span><span class=\"TextRun SCXW57888485 BCX0\"><span class=\"NormalTextRun SpellingErrorV2Themed SCXW57888485 BCX0\">level<\/span><span class=\"NormalTextRun SCXW57888485 BCX0\">\u00a0control and can view unfiltered data.<\/span><span class=\"NormalTextRun SCXW57888485 BCX0\">\u00a0Also, i<\/span><span class=\"NormalTextRun SCXW57888485 BCX0\">f a table has\u00a0<\/span><\/span><span class=\"TextRun SCXW57888485 BCX0\"><span class=\"NormalTextRun SCXW57888485 BCX0\">no<\/span><\/span><span class=\"TextRun SCXW57888485 BCX0\"><span class=\"NormalTextRun SCXW57888485 BCX0\">\u00a0RLS policy, users see full unfiltered data<\/span><span class=\"NormalTextRun SCXW57888485 BCX0\">.<\/span><\/span><\/p><\/div>\n\t\t\t\t\t<\/div><div class=\"eael-accordion-list\">\n\t\t\t\t\t<div id=\"what-is-the-query-to-implement-rls-on-the-data-in-a-particular-table\" class=\"elementor-tab-title eael-accordion-header\" tabindex=\"0\" data-tab=\"17\" aria-controls=\"elementor-tab-content-93417\"><span class=\"eael-advanced-accordion-icon-closed\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-plus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H272V64c0-17.67-14.33-32-32-32h-32c-17.67 0-32 14.33-32 32v144H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h144v144c0 17.67 14.33 32 32 32h32c17.67 0 32-14.33 32-32V304h144c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-advanced-accordion-icon-opened\"><svg aria-hidden=\"true\" class=\"fa-accordion-icon e-font-icon-svg e-fas-minus\" viewBox=\"0 0 448 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M416 208H32c-17.67 0-32 14.33-32 32v32c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32v-32c0-17.67-14.33-32-32-32z\"><\/path><\/svg><\/span><span class=\"eael-accordion-tab-title\">What is the query to implement RLS on the data in a particular table?<\/span><svg aria-hidden=\"true\" class=\"fa-toggle e-font-icon-svg e-fas-angle-right\" viewBox=\"0 0 256 512\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\"><path d=\"M224.3 273l-136 136c-9.4 9.4-24.6 9.4-33.9 0l-22.6-22.6c-9.4-9.4-9.4-24.6 0-33.9l96.4-96.4-96.4-96.4c-9.4-9.4-9.4-24.6 0-33.9L54.3 103c9.4-9.4 24.6-9.4 33.9 0l136 136c9.5 9.4 9.5 24.6.1 34z\"><\/path><\/svg><\/div><div id=\"elementor-tab-content-93417\" class=\"eael-accordion-content clearfix\" data-tab=\"17\" aria-labelledby=\"what-is-the-query-to-implement-rls-on-the-data-in-a-particular-table\"><p><span>&#8211; &#8211; Creating inline table-valued function &#8211; &#8211;<\/span><span>\u00a0<\/span><\/p><p><span>Alter FUNCTION\u00a0Security.country_securitypredicate(Country AS\u00a0nvarchar(50))<\/span><span>\u00a0<\/span><\/p><p><span>\u202f \u202f RETURNS TABLE<\/span><span>\u00a0<\/span><\/p><p><span>WITH SCHEMABINDING<\/span><span>\u00a0<\/span><\/p><p><span>AS<\/span><span>\u00a0<\/span><\/p><p><span>\u202f \u202f RETURN SELECT 1 AS\u00a0country_securitypredicate_result<\/span><span>\u00a0<\/span><\/p><p><span>WHERE @Country = &#8216;Germany&#8217; AND USER_NAME() = &#8216;sabhye.singhal@synlab.com&#8217;;<\/span><span>\u00a0<\/span><\/p><p><span>GO<\/span><span>\u00a0<\/span><\/p><p><span>\u202f<\/span><span>\u00a0<\/span><\/p><p><span>&#8211; &#8211; Using the function to create a Security Policy &#8211; &#8211;<\/span><span>\u00a0<\/span><\/p><p><span>CREATE SECURITY POLICY\u00a0filter_based_on_country<\/span><span>\u00a0<\/span><\/p><p><span>ADD FILTER PREDICATE\u00a0Security.country_securitypredicate(NOM_AGENCE)<\/span><span>\u00a0<\/span><\/p><p><span>ON\u00a0dbo.pxo_validated_tickets;<\/span><span>\u00a0<\/span><\/p><p><span>GO<\/span><span>\u00a0<\/span><\/p><p><span>\u00a0<\/span><\/p><p><span>&#8211; &#8211; Revoking the security policy &#8211; &#8211;<\/span><span>\u00a0<\/span><\/p><p><span>Drop security policy\u00a0filter_based_on_country<\/span><span>\u00a0<\/span><\/p><\/div>\n\t\t\t\t\t<\/div><\/div>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Medallion Architecture in Microsoft Fabric: From Ingestion to Secured Reporting A practical, end-to-end walkthrough of building the bronze, silver, and gold layers in Microsoft Fabric, including data masking and row-level security in the gold layer. Medallion architecture diagram in Microsoft Fabric: data sources flow via pipelines and shortcuts into Bronze, Silver and Gold layers in [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":23322,"comment_status":"open","ping_status":"open","sticky":false,"template":"elementor_header_footer","format":"standard","meta":{"footnotes":""},"categories":[236,51],"tags":[250,253,251,255,249,254,252],"class_list":["post-23311","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-integration-and-data-engineering","category-technology-trends","tag-bronze-silver-gold-layers","tag-data-masking-microsoft-fabric","tag-dataflow-gen2-vs-shortcuts","tag-fabric-lakehouse-to-warehouse","tag-fabric-medallion-architecture","tag-row-level-security-fabric-warehouse","tag-sql-server-mirroring-fabric"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v27.8 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Medallion Architecture in Microsoft Fabric: A Guide<\/title>\n<meta name=\"description\" content=\"A step-by-step guide to building medallion architecture in Microsoft Fabric: bronze, silver, and gold layers, plus data masking and row-level security.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Medallion Architecture in Microsoft Fabric: A Guide\" \/>\n<meta property=\"og:description\" content=\"A step-by-step guide to building medallion architecture in Microsoft Fabric: bronze, silver, and gold layers, plus data masking and row-level security.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/\" \/>\n<meta property=\"og:site_name\" content=\"Expert Extend\" \/>\n<meta property=\"article:published_time\" content=\"2026-07-10T04:43:31+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-07-16T06:42:39+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/ChatGPT-Image-Jul-9-2026-11_39_18-AM.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1536\" \/>\n\t<meta property=\"og:image:height\" content=\"1024\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"expert\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"expert\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"25 minutes\" \/>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Medallion Architecture in Microsoft Fabric: A Guide","description":"A step-by-step guide to building medallion architecture in Microsoft Fabric: bronze, silver, and gold layers, plus data masking and row-level security.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/","og_locale":"en_US","og_type":"article","og_title":"Medallion Architecture in Microsoft Fabric: A Guide","og_description":"A step-by-step guide to building medallion architecture in Microsoft Fabric: bronze, silver, and gold layers, plus data masking and row-level security.","og_url":"https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/","og_site_name":"Expert Extend","article_published_time":"2026-07-10T04:43:31+00:00","article_modified_time":"2026-07-16T06:42:39+00:00","og_image":[{"width":1536,"height":1024,"url":"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/ChatGPT-Image-Jul-9-2026-11_39_18-AM.png","type":"image\/png"}],"author":"expert","twitter_card":"summary_large_image","twitter_misc":{"Written by":"expert","Est. reading time":"25 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/#article","isPartOf":{"@id":"https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/"},"author":{"name":"expert","@id":"https:\/\/expertextend.de\/en\/#\/schema\/person\/84fe20b71bfd5073872f716f49f6884f"},"headline":"Medallion Architecture In Fabric","datePublished":"2026-07-10T04:43:31+00:00","dateModified":"2026-07-16T06:42:39+00:00","mainEntityOfPage":{"@id":"https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/"},"wordCount":3917,"commentCount":0,"publisher":{"@id":"https:\/\/expertextend.de\/en\/#organization"},"image":{"@id":"https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/#primaryimage"},"thumbnailUrl":"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/ChatGPT-Image-Jul-9-2026-11_39_18-AM.png","keywords":["bronze silver gold layers","data masking Microsoft Fabric","Dataflow Gen2 vs shortcuts","Fabric lakehouse to warehouse","Fabric medallion architecture","row-level security Fabric warehouse","SQL Server mirroring Fabric"],"articleSection":["Data Integration and Data Engineering","Technology Trends"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/","url":"https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/","name":"Medallion Architecture in Microsoft Fabric: A Guide","isPartOf":{"@id":"https:\/\/expertextend.de\/en\/#website"},"primaryImageOfPage":{"@id":"https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/#primaryimage"},"image":{"@id":"https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/#primaryimage"},"thumbnailUrl":"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/ChatGPT-Image-Jul-9-2026-11_39_18-AM.png","datePublished":"2026-07-10T04:43:31+00:00","dateModified":"2026-07-16T06:42:39+00:00","description":"A step-by-step guide to building medallion architecture in Microsoft Fabric: bronze, silver, and gold layers, plus data masking and row-level security.","breadcrumb":{"@id":"https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/#primaryimage","url":"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/ChatGPT-Image-Jul-9-2026-11_39_18-AM.png","contentUrl":"https:\/\/expertextend.de\/wp-content\/uploads\/2026\/07\/ChatGPT-Image-Jul-9-2026-11_39_18-AM.png","width":1536,"height":1024},{"@type":"BreadcrumbList","@id":"https:\/\/expertextend.de\/en\/medallion-architecture-microsoft-fabric\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/expertextend.de\/en\/"},{"@type":"ListItem","position":2,"name":"Medallion Architecture In Fabric"}]},{"@type":"WebSite","@id":"https:\/\/expertextend.de\/en\/#website","url":"https:\/\/expertextend.de\/en\/","name":"Expert Extend","description":"DATA \u00b7 INSIGHTS \u00b7 ACTION","publisher":{"@id":"https:\/\/expertextend.de\/en\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/expertextend.de\/en\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/expertextend.de\/en\/#organization","name":"Expert Extend GMBH","url":"https:\/\/expertextend.de\/en\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/expertextend.de\/en\/#\/schema\/logo\/image\/","url":"https:\/\/expertextend.de\/wp-content\/uploads\/2025\/03\/Expert-Extend-Logo-Symbol.png","contentUrl":"https:\/\/expertextend.de\/wp-content\/uploads\/2025\/03\/Expert-Extend-Logo-Symbol.png","width":1200,"height":1200,"caption":"Expert Extend GMBH"},"image":{"@id":"https:\/\/expertextend.de\/en\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.linkedin.com\/company\/expertextend\/"]},{"@type":"Person","@id":"https:\/\/expertextend.de\/en\/#\/schema\/person\/84fe20b71bfd5073872f716f49f6884f","name":"expert","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/0145afd5c1943f02f47435d07895d78e923ca3aa3267d287f9b80286f5150b9f?s=96&d=mm&r=g","url":"https:\/\/secure.gravatar.com\/avatar\/0145afd5c1943f02f47435d07895d78e923ca3aa3267d287f9b80286f5150b9f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/0145afd5c1943f02f47435d07895d78e923ca3aa3267d287f9b80286f5150b9f?s=96&d=mm&r=g","caption":"expert"},"sameAs":["https:\/\/expertextend.de"],"url":"https:\/\/expertextend.de\/en\/author\/expert\/"}]}},"_links":{"self":[{"href":"https:\/\/expertextend.de\/en\/wp-json\/wp\/v2\/posts\/23311","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/expertextend.de\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/expertextend.de\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/expertextend.de\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/expertextend.de\/en\/wp-json\/wp\/v2\/comments?post=23311"}],"version-history":[{"count":154,"href":"https:\/\/expertextend.de\/en\/wp-json\/wp\/v2\/posts\/23311\/revisions"}],"predecessor-version":[{"id":23562,"href":"https:\/\/expertextend.de\/en\/wp-json\/wp\/v2\/posts\/23311\/revisions\/23562"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/expertextend.de\/en\/wp-json\/wp\/v2\/media\/23322"}],"wp:attachment":[{"href":"https:\/\/expertextend.de\/en\/wp-json\/wp\/v2\/media?parent=23311"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/expertextend.de\/en\/wp-json\/wp\/v2\/categories?post=23311"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/expertextend.de\/en\/wp-json\/wp\/v2\/tags?post=23311"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}