Watermark Table

Watermark Table: What You Should Know

The watermark is added on-premises to the source table with one key called watermark_key. Each run adds a new row to this key of the  watermark column. With multiple updates, the watermark row is added to the key for each new run; therefore, multiple  watermark value row must be added at a time.  Note: Watermark  is not an updating key in Azure Data Factory. Instead, watermark_key is the only key that contains the date value associated with each entry in the high-watermark table. This update  value is not referenced in the update output of the source data.  Azure Data Factory and the High-Watermark table — Example —  Note: watermark_key is the only key that contains the date attribute, which is the watermark_time_stamp_value_key in the  Example Table.  You can also use an alternate key, watermark_value_key. An alternative key may include a column to store the incrementing key for the watermark value row. Example —  Incremental Data Loading Using Azure Data Factory — Example Table — Azure Data Factory  Incremental Data Loading with a control table — Example Table Incremental Data Loading in Azure SQL Database — Example Table Azure Data Factory + control table = Incremental Data Loading in Azure SQL Database Using Azure Data Factory + Azure data warehouse table = Incremental Data Loading in Azure SQL DB See, incremental loading is so easy. Once an Incremental Loading data is  presented to it, it can do all the heavy lifting itself and do it on-premises using on Azure watermark store.