From ff3cf92d0466c423dc479078b7fa3d5d107fcc7e Mon Sep 17 00:00:00 2001
From: Yam ZhengLim <yamzhenglim0427@gmail.com>
Date: Thu, 30 Mar 2023 10:31:27 +0000
Subject: [PATCH 1/2] Readme update

---
 README.md | 87 +++++++++++++++++++++++++++++++------------------------
 1 file changed, 49 insertions(+), 38 deletions(-)

diff --git a/README.md b/README.md
index d24c02a..985b516 100644
--- a/README.md
+++ b/README.md
@@ -32,56 +32,82 @@ Short naming converntion lists that need to follow
 <br/>
 
 ## DBT Rule
-### Structure of the dbt folder model
+### Structure of the dbt model layer
 - Analyses
-    1. Analyses model will not create in Snowflake
+    1. Analyses layer will not create in Snowflake
     2. You can write some adhoc queries 
-    3. test some SQL code before write in Models
+    3. Test some SQL code before write in Models
 
 <br/>
 
 - Macros
-    1. Jinja function that reuse in other .sql / .py file
+    1. Jinja function that reuse in other models
 
-<br>
+<br/>
 
 - Models 
     - Staging
-        1. Materialise: `View`  
-        2. 1-to-1 relationship (or mapping) to source tables.
-        3. Column renaming
-        4. Column remove
-        5. Column data renaming, such as (status integer to string)
-        6. Data input error cleansing
-        7. Check for raw data freshness
-        8. Data type transformation
+        - System name
+            1. Materialise: `View`  
+            2. 1-to-1 relationship (or mapping) to source tables.
+            3. Column renaming
+            4. Column remove
+            5. Column data renaming, such as (status integer to string)
+            6. Data input error cleansing
+            7. Check for raw data freshness
+            8. Data type transformation
+            9. Data split and merge
+            - Source
+                1. Source will write in YML
+                2. Check Freshness 
+                3. Check duplicate id
+                4. Check null id
 
     - Intermediate
         1. Materialise: `Ephemerally`
         2. Stacking layers of logic with clear and specific purposes to prepa our staging models to join into the entities we want
         3. Be referenced repeatedly in more than one model
-        4. Isolating complex operations
+        4. Isolating complex operations       
         
-        
-    - marts/warehouse
-        1. Store fact and dimension models
+    - Marts/warehouse
+        1. Materialise: `Table`
+        2. Store fact and dimension models
 
-    - marts/reporting
-        1. Store custom reports
+    - Marts/reporting
+        1. Materialise: `Table`
+        2. Store custom reports
+
+<br/>
 
 - Seeds
+    1. Store custom dataset in csv format
+    2. The dataset must not change frequently
+
+<br/>
 
 - Snapshots
+    1. DBT build-in SCD type 2 function
+    2. Prefer to use after the source table and before the staging layer
+
+<br/>
 
 - Tests
+    1. Can write some yml test case
+
+<br/>
 
 ### File Naming Rules
 - File names must be unique
 - Each sql/python file must start with [`model_type`]_[`system_name`]__[`model_name`]s.sql/py
-- The model should be name as plural, (eg: stg_exchange__orders.sql)
+- The model should be name as plural. (eg: stg_exchange__orders.sql)
+
+<br/>
 
 ### Column Naming Rules
-- If array datatype, the column naming must be plural eg (ids, messages)
+- If array datatype, the column naming must be plural (eg: ids, messages)
+- all the date & time field will need to name as datetime / date (eg: created_datetime)
+
+<br/>
 
 ### SQL Coding Rule
 - The SQL clause **must** be UPPERCASE (eg. `SELECT`, `FROM`, `WHERE`, `GROUP BY`, `LIMIT`, `WITH`, `AS`, `SUM`, `PARTITION OVER`, `DIV0`, `LEFT JOIN`, `ON` ....)
@@ -96,7 +122,7 @@ WITH [table_names] AS (
 ),
 
 --LOGIC
-[logic_name] AS (
+[logic_names] AS (
     .....
 ),
 
@@ -104,21 +130,6 @@ WITH [table_names] AS (
 final__[table_names] AS (
 
 )
-SELECT * FROM final__[table_names]
+SELECT * FROM final__[final_table_names]
 ```
 
-
-
-
-
-Try running the following commands:
-- dbt run
-- dbt test
-
-
-### Resources:
-- Learn more about dbt [in the docs](https://docs.getdbt.com/docs/introduction)
-- Check out [Discourse](https://discourse.getdbt.com/) for commonly asked questions and answers
-- Join the [dbt community](http://community.getbdt.com/) to learn from other analytics engineers
-- Find [dbt events](https://events.getdbt.com) near you
-- Check out [the blog](https://blog.getdbt.com/) for the latest news on dbt's development and best practices

From 691d25a2004d7d0028aec6afbfc9907f2abe2637 Mon Sep 17 00:00:00 2001
From: Yam ZhengLim <yamzhenglim0427@gmail.com>
Date: Thu, 30 Mar 2023 10:32:40 +0000
Subject: [PATCH 2/2] Readme update

---
 README.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/README.md b/README.md
index 985b516..cb18907 100644
--- a/README.md
+++ b/README.md
@@ -47,7 +47,7 @@ Short naming converntion lists that need to follow
 
 - Models 
     - Staging
-        - System name
+        - [system_name]
             1. Materialise: `View`  
             2. 1-to-1 relationship (or mapping) to source tables.
             3. Column renaming