diff --git a/README.md b/README.md
index d24c02a..ae37e92 100644
--- a/README.md
+++ b/README.md
@@ -32,56 +32,129 @@ Short naming converntion lists that need to follow
## DBT Rule
-### Structure of the dbt folder model
+### Structure of the dbt model layer
- Analyses
- 1. Analyses model will not create in Snowflake
+ 1. Analyses layer will not create in Snowflake
2. You can write some adhoc queries
- 3. test some SQL code before write in Models
+ 3. Test some SQL code before write in Models
- Macros
- 1. Jinja function that reuse in other .sql / .py file
+ 1. Jinja function that reuse in other models
-
+
- Models
- Staging
- 1. Materialise: `View`
- 2. 1-to-1 relationship (or mapping) to source tables.
- 3. Column renaming
- 4. Column remove
- 5. Column data renaming, such as (status integer to string)
- 6. Data input error cleansing
- 7. Check for raw data freshness
- 8. Data type transformation
+ - System name
+ 1. Materialise: `View`
+ 2. 1-to-1 relationship (or mapping) to source tables.
+ 3. Column renaming
+ 4. Column remove
+ 5. Column data renaming, such as (status integer to string)
+ 6. Data input error cleansing
+ 7. Check for raw data freshness
+ 8. Data type transformation
+ 9. Data split and merge
+ - Source
+ 1. Source will write in YML
+ 2. Check Freshness
+ 3. Check duplicate id
+ 4. Check null id
- Intermediate
1. Materialise: `Ephemerally`
2. Stacking layers of logic with clear and specific purposes to prepa our staging models to join into the entities we want
3. Be referenced repeatedly in more than one model
- 4. Isolating complex operations
+ 4. Isolating complex operations
-
- - marts/warehouse
- 1. Store fact and dimension models
+ - Marts/warehouse
+ 1. Materialise: `Table`
+ 2. Store fact and dimension models
- - marts/reporting
- 1. Store custom reports
+ - Marts/reporting
+ 1. Materialise: `Table`
+ 2. Store custom reports
+
+
- Seeds
+ 1. Store custom dataset in csv format
+ 2. The dataset must not change frequently
+
+
- Snapshots
+ 1. DBT build-in SCD type 2 function
+ 2. Prefer to use after the source table and before the staging layer
+
+
- Tests
+ 1. Can write some yml test case
+
+
### File Naming Rules
- File names must be unique
- Each sql/python file must start with [`model_type`]_[`system_name`]__[`model_name`]s.sql/py
-- The model should be name as plural, (eg: stg_exchange__orders.sql)
+- The model should be name as plural. (eg: stg_exchange__orders.sql)
+
+
### Column Naming Rules
-- If array datatype, the column naming must be plural eg (ids, messages)
+- If array datatype, the column naming must be plural (eg: ids, messages)
+- Schema, table and column names should be in `snake_case`.
+
+- Limit use of abbreviations that are related to domain knowledge. An onboarding
+ employee will understand `current_order_status` better than `current_os`.
+
+- Use names based on the _business_ terminology, rather than the source terminology.
+
+- Each model should have a primary key that can identify the unique row, and should be named `_id`, e.g. `account_id` – this makes it easier to know what `id` is being referenced in downstream joined models.
+
+- If a surrogate key is created, it should be named `_sk`.
+
+- For `base` or `staging` models, columns should be ordered in categories, where identifiers are first and date/time fields are at the end.
+ Example:
+ ```sql
+ transformed as (
+ select
+ -- ids
+ order_id,
+ customer_id,
+
+ -- dimensions
+ order_status,
+ is_shipped,
+
+ -- measures
+ order_total,
+
+ -- date/times
+ created_at,
+ updated_at,
+
+ -- metadata
+ _sdc_batched_at
+ from source
+ )
+ ```
+
+- Date/time columns should be named according to these conventions:
+ - Timestamps: `_datetime`
+ Example: `created_datetime`
+
+ - Dates: `_date`
+ Example: `created_date`
+
+- Booleans should be prefixed with `is_` or `has_`.
+ Example: `is_active_customer` and `has_admin_access`
+
+- Price/revenue fields should be in decimal currency (e.g. `19.99` for $19.99; many app databases store prices as integers in cents). If non-decimal currency is used, indicate this with suffix, e.g. `price_in_cents`.
+
+
### SQL Coding Rule
- The SQL clause **must** be UPPERCASE (eg. `SELECT`, `FROM`, `WHERE`, `GROUP BY`, `LIMIT`, `WITH`, `AS`, `SUM`, `PARTITION OVER`, `DIV0`, `LEFT JOIN`, `ON` ....)
@@ -96,7 +169,7 @@ WITH [table_names] AS (
),
--LOGIC
-[logic_name] AS (
+[logic_names] AS (
.....
),
@@ -104,21 +177,8 @@ WITH [table_names] AS (
final__[table_names] AS (
)
-SELECT * FROM final__[table_names]
+SELECT * FROM final__[final_table_names]
```
-
-
-
-
-Try running the following commands:
-- dbt run
-- dbt test
-
-
-### Resources:
-- Learn more about dbt [in the docs](https://docs.getdbt.com/docs/introduction)
-- Check out [Discourse](https://discourse.getdbt.com/) for commonly asked questions and answers
-- Join the [dbt community](http://community.getbdt.com/) to learn from other analytics engineers
-- Find [dbt events](https://events.getdbt.com) near you
-- Check out [the blog](https://blog.getdbt.com/) for the latest news on dbt's development and best practices
+### Testing
+- At a minimum, `unique` and `not_null` tests should be applied to the expected primary key of each model.
\ No newline at end of file
diff --git a/analyses/exchange/company_conversion_v2.sql b/analyses/exchange/company_conversion_v2.sql
index af2afe3..e124bcb 100644
--- a/analyses/exchange/company_conversion_v2.sql
+++ b/analyses/exchange/company_conversion_v2.sql
@@ -1,5 +1,5 @@
-- MARCRO for count the cummulative_activity_number
-{% set funnel_activity = ['register','email_verified','identity_documents_uploaded','identity_documents_verified','first_time_booking','first_time_order','first_time_order_completed','repeat_completed_order_after_30days','repeat_completed_order_after_60days']%}
+{% set funnel_activity = ['register','email_verified','identity_document_uploaded','identity_document_verified','first_time_booking','first_time_order','first_time_order_completed','repeat_completed_order_after_30days','repeat_completed_order_after_60days']%}
-- IMPORT
WITH companies AS (
@@ -69,12 +69,13 @@ company_first_time_order_date AS (
---start defind funnel
+--start defind funnel activity
repeat_completed_order_after_60days_activities AS (
SELECT
clean_company_transaction_orders.company_id,
0 AS is_data_cleansing_generate_row,
'repeat_completed_order_after_60days' AS activity,
+ 9 AS activtity_rank,
MIN(clean_company_transaction_orders.created_at) AS activity_datetime
FROM clean_company_transaction_orders
@@ -94,6 +95,7 @@ repeat_completed_order_after_30days_activities AS (
clean_company_transaction_orders.company_id,
0 AS is_data_cleansing_generate_row,
'repeat_completed_order_after_30days' AS activity,
+ 8 AS activtity_rank,
MIN(clean_company_transaction_orders.created_at) AS activity_datetime
FROM clean_company_transaction_orders
@@ -113,6 +115,7 @@ first_time_order_completed_activities AS (
clean_company_transaction_orders.company_id,
0 AS is_data_cleansing_generate_row,
'first_time_order_completed' AS activity,
+ 7 AS activtity_rank,
MIN(clean_company_transaction_orders.created_at) AS activity_datetime
FROM clean_company_transaction_orders
@@ -129,6 +132,7 @@ first_time_order_activities AS (
clean_company_transaction_orders.company_id,
0 AS is_data_cleansing_generate_row,
'first_time_order' AS activity,
+ 6 AS activtity_rank,
MIN(clean_company_transaction_orders.created_at) AS activity_datetime
FROM clean_company_transaction_orders
@@ -145,6 +149,7 @@ first_time_booking_activities AS (
IFNULL( unclean_first_time_booking_activities.company_id , first_time_order_activities.company_id ) AS company_id,
IFF( unclean_first_time_booking_activities.company_id IS NULL, 1, 0) AS is_data_cleansing_generate_row,
'first_time_booking' AS activity,
+ 5 AS activtity_rank,
IFNULL( unclean_first_time_booking_activities.activity_datetime , first_time_order_activities.activity_datetime ) AS activity_datetime
FROM
@@ -168,19 +173,20 @@ first_time_booking_activities AS (
),
-identity_documents_verified_activities AS (
+identity_document_verified_activities AS (
SELECT
- IFNULL( unclean_identity_documents_verified_activities.company_id , first_time_booking_activities.company_id ) AS company_id,
- IFF( unclean_identity_documents_verified_activities.company_id IS NULL, 1, 0) AS is_data_cleansing_generate_row,
- 'identity_documents_verified' AS activity,
- IFNULL( unclean_identity_documents_verified_activities.activity_datetime , first_time_booking_activities.activity_datetime ) AS activity_datetime
+ IFNULL( unclean_identity_document_verified_activities.company_id , first_time_booking_activities.company_id ) AS company_id,
+ IFF( unclean_identity_document_verified_activities.company_id IS NULL, 1, 0) AS is_data_cleansing_generate_row,
+ 'identity_document_verified' AS activity,
+ 4 AS activtity_rank,
+ IFNULL( unclean_identity_document_verified_activities.activity_datetime , first_time_booking_activities.activity_datetime ) AS activity_datetime
FROM
(
SELECT
company_id,
- 'identity_documents_verified' AS activity,
+ 'identity_document_verified' AS activity,
MIN(identity_created_datetime) AS activity_datetime
FROM company_identity_documents
@@ -191,27 +197,28 @@ identity_documents_verified_activities AS (
GROUP BY
company_id
- ) AS unclean_identity_documents_verified_activities
+ ) AS unclean_identity_document_verified_activities
FULL OUTER JOIN first_time_booking_activities
- ON (unclean_identity_documents_verified_activities.company_id = first_time_booking_activities.company_id)
+ ON (unclean_identity_document_verified_activities.company_id = first_time_booking_activities.company_id)
),
-identity_documents_uploaded_activities AS (
+identity_document_uploaded_activities AS (
SELECT
- IFNULL( unclean_identity_documents_uploaded_activities.company_id , identity_documents_verified_activities.company_id ) AS company_id,
- IFF( unclean_identity_documents_uploaded_activities.company_id IS NULL, 1, 0) AS is_data_cleansing_generate_row,
- 'identity_documents_uploaded' AS activity,
- IFNULL( unclean_identity_documents_uploaded_activities.activity_datetime , identity_documents_verified_activities.activity_datetime ) AS activity_datetime
+ IFNULL( unclean_identity_document_uploaded_activities.company_id , identity_document_verified_activities.company_id ) AS company_id,
+ IFF( unclean_identity_document_uploaded_activities.company_id IS NULL, 1, 0) AS is_data_cleansing_generate_row,
+ 'identity_document_uploaded' AS activity,
+ 3 AS activtity_rank,
+ IFNULL( unclean_identity_document_uploaded_activities.activity_datetime , identity_document_verified_activities.activity_datetime ) AS activity_datetime
FROM
(
SELECT
company_id,
- 'identity_documents_uploaded' AS activity,
+ 'identity_document_uploaded' AS activity,
MIN(identity_created_datetime) AS activity_datetime
FROM company_identity_documents
@@ -219,19 +226,20 @@ identity_documents_uploaded_activities AS (
GROUP BY
company_id
- ) AS unclean_identity_documents_uploaded_activities
+ ) AS unclean_identity_document_uploaded_activities
- FULL OUTER JOIN identity_documents_verified_activities
- ON (unclean_identity_documents_uploaded_activities.company_id = identity_documents_verified_activities.company_id)
+ FULL OUTER JOIN identity_document_verified_activities
+ ON (unclean_identity_document_uploaded_activities.company_id = identity_document_verified_activities.company_id)
),
email_verified_activities AS (
SELECT
- IFNULL( unclean_email_verified_activities.company_id , identity_documents_uploaded_activities.company_id ) AS company_id,
+ IFNULL( unclean_email_verified_activities.company_id , identity_document_uploaded_activities.company_id ) AS company_id,
IFF( unclean_email_verified_activities.company_id IS NULL, 1, 0) AS is_data_cleansing_generate_row,
'email_verified' AS activity,
- IFNULL( unclean_email_verified_activities.activity_datetime , identity_documents_uploaded_activities.activity_datetime ) AS activity_datetime
+ 2 AS activtity_rank,
+ IFNULL( unclean_email_verified_activities.activity_datetime , identity_document_uploaded_activities.activity_datetime ) AS activity_datetime
FROM
(
@@ -249,8 +257,8 @@ email_verified_activities AS (
company_id
) AS unclean_email_verified_activities
- FULL OUTER JOIN identity_documents_uploaded_activities
- ON (unclean_email_verified_activities.company_id = identity_documents_uploaded_activities.company_id)
+ FULL OUTER JOIN identity_document_uploaded_activities
+ ON (unclean_email_verified_activities.company_id = identity_document_uploaded_activities.company_id)
),
@@ -259,6 +267,7 @@ register_activities AS (
IFNULL( unclean_register_activities.company_id , email_verified_activities.company_id ) AS company_id,
IFF( unclean_register_activities.company_id IS NULL, 1, 0) AS is_data_cleansing_generate_row,
'register' AS activity,
+ 1 AS activtity_rank,
IFNULL( unclean_register_activities.activity_datetime , email_verified_activities.activity_datetime ) AS activity_datetime
FROM
@@ -310,24 +319,30 @@ union_all_activities_flag_migrated_company AS (
--FINAL
-sem_exchange__company_funnels AS (
+sem_exchange__company_conversion_funnels AS (
SELECT
company_id,
is_data_cleansing_generate_row,
is_migrated_company,
+ activity,
+ activtity_rank,
+ IFF( ROW_NUMBER() OVER (PARTITION BY company_id ORDER BY activtity_rank DESC) = 1,
+ 1,
+ 0) AS latest_activity,
+
activity_datetime,
- {% for activity in funnel_activity %}
+ {% for activity_type in funnel_activity %}
- count(iff(activity = '{{activity}}', company_id, NULL))
- OVER (ORDER BY activity_datetime) AS cumulative_{{activity}}_activity,
+ count(iff(activity = '{{activity_type}}', company_id, NULL))
+ OVER (ORDER BY activity_datetime) AS cumulative_{{activity_type}}_activity,
{% endfor %}
- {% for activity in funnel_activity %}
+ {% for activity_type in funnel_activity %}
- count(iff(activity = '{{activity}}' AND is_migrated_company = 0, company_id, NULL))
- OVER (ORDER BY activity_datetime) AS exclude_migrated_cumulative_{{activity}}_activity,
+ count(iff(activity = '{{activity_type}}' AND is_migrated_company = 0, company_id, NULL))
+ OVER (ORDER BY activity_datetime) AS exclude_migrated_cumulative_{{activity_type}}_activity,
{% endfor %}
@@ -338,4 +353,4 @@ sem_exchange__company_funnels AS (
ORDER BY activity_datetime
)
-SELECT * FROM sem_exchange__company_funnels
\ No newline at end of file
+SELECT * FROM sem_exchange__company_conversion_funnels
\ No newline at end of file
diff --git a/models/exchange/staging/activity_logs/stg_exchange__activity_logs.sql b/models/exchange/staging/activity_logs/stg_exchange__activity_logs.sql
deleted file mode 100644
index 3e83c4a..0000000
--- a/models/exchange/staging/activity_logs/stg_exchange__activity_logs.sql
+++ /dev/null
@@ -1,46 +0,0 @@
--- IMPORTS
-WITH activity_logs AS (
- SELECT * FROM {{ ref('base_exchange__activity_logs') }}
-),
-
-
--- LOGIC
-activity_logs_listout AS (
-
- SELECT
- activity_logs.id,
- activity_logs.log_name,
- activity_logs.description,
- activity_logs.subject_id,
- activity_logs.subject_type,
- activity_logs.causer_id,
- activity_logs.causer_type,
- activity_logs.properties,
- activity_logs.created_at,
- activity_logs.updated_at,
- CURRENT_TIMESTAMP() AS _dbt_ran_at
-
- FROM activity_logs
-),
-
-
--- FINAL
-final__stg_exchange__activity_logs AS (
-
- SELECT
- id,
- log_name,
- description,
- subject_id,
- subject_type,
- causer_id,
- causer_type,
- properties,
- created_at,
- updated_at,
- _dbt_ran_at
-
- FROM activity_logs_listout
-)
-
-SELECT * FROM final__stg_exchange__activity_logs
\ No newline at end of file
diff --git a/models/exchange/staging/wallet/stg_exchange__wallet.sql b/models/exchange/staging/wallet/stg_exchange__wallet.sql
new file mode 100644
index 0000000..60fdb61
--- /dev/null
+++ b/models/exchange/staging/wallet/stg_exchange__wallet.sql
@@ -0,0 +1,10 @@
+-- IMPORTS
+WITH wallets AS (
+ SELECT * FROM {{ ref('base_exchange__wallets') }}
+)
+
+
+
+
+-- FINAL
+SELECT * FROM wallets
\ No newline at end of file
diff --git a/models/exchange/staging/wallet_logs/stg_exchange__wallet_logs.sql b/models/exchange/staging/wallet_logs/stg_exchange__wallet_logs.sql
new file mode 100644
index 0000000..2e0ad6f
--- /dev/null
+++ b/models/exchange/staging/wallet_logs/stg_exchange__wallet_logs.sql
@@ -0,0 +1,7 @@
+-- IMPORTS
+with wallet_logs as (select * from {{ ref("base_exchange__wallet_logs") }})
+
+-- FINAL
+select *
+from wallet_logs
+order by wallet_id, updated_at asc
diff --git a/models/shipping/staging/stg_shipping__activity_logs.sql b/models/shipping/staging/stg_shipping__activity_logs.sql
new file mode 100644
index 0000000..8f30a63
--- /dev/null
+++ b/models/shipping/staging/stg_shipping__activity_logs.sql
@@ -0,0 +1,46 @@
+-- IMPORTS
+WITH activity_logs AS (
+ SELECT * FROM {{ source('src_shipping_mysql', 'activity_log') }}
+),
+
+
+-- LOGIC
+activity_logs_rename AS (
+
+ SELECT
+ activity_logs.id,
+ activity_logs.log_name,
+ activity_logs.description,
+ activity_logs.subject_id,
+ activity_logs.subject_type,
+ activity_logs.causer_id,
+ activity_logs.causer_type,
+ activity_logs.properties,
+ activity_logs.created_at AS created_datetime,
+ activity_logs.updated_at AS updated_datetime,
+ current_timestamp() AS _dbt_ran_at
+
+ FROM activity_logs
+),
+
+
+-- FINAL
+final__base_exchange__activity_logs AS (
+
+ SELECT
+ id,
+ log_name,
+ description,
+ subject_id,
+ subject_type,
+ causer_id,
+ causer_type,
+ properties,
+ created_datetime,
+ updated_datetime,
+ _dbt_ran_at
+
+ FROM activity_logs_rename
+)
+
+SELECT * FROM final__base_exchange__activity_logs
\ No newline at end of file
diff --git a/models/staging/autocount/_autocount__sources.yml b/models/staging/autocount/_autocount__sources.yml
new file mode 100644
index 0000000..e69de29
diff --git a/models/staging/crisp/_crisp__sources.yml b/models/staging/crisp/_crisp__sources.yml
new file mode 100644
index 0000000..e69de29
diff --git a/models/staging/exchange/_exchange__docs.md b/models/staging/exchange/_exchange__docs.md
new file mode 100644
index 0000000..ed501d3
--- /dev/null
+++ b/models/staging/exchange/_exchange__docs.md
@@ -0,0 +1,23 @@
+{% docs src_exchange_mysql %}
+
+Data is extract from CIEF Selfhost MYSQL `exchange_production`, using custom python code.
+
+- `_source_loaded_at` will add into the raw data when extract
+```YML
+loaded_at_field: _source_loaded_at
+freshness:
+ warn_after: {count: 12, period: hour}
+ error_after: {count: 24, period: hour}
+```
+
+- The raw data will use copy into snowflake in 2AM everyday
+
+- Depend on target.name to determin which snowflake raw schema
+
+```JINJA
+{%- if target.name == "prod" -%} PROD_CIEF_RAW_DB
+{%- else -%} DEV_CIEF_RAW_DB
+{%- endif -%}
+```
+
+{% enddocs %}
\ No newline at end of file
diff --git a/models/staging/exchange/_exchange__models.yml b/models/staging/exchange/_exchange__models.yml
new file mode 100644
index 0000000..cc0f92b
--- /dev/null
+++ b/models/staging/exchange/_exchange__models.yml
@@ -0,0 +1,6 @@
+version: 2
+
+models:
+ - name: stg_exchange__activity_logs
+
+ - name: stg_exchange__announcments
\ No newline at end of file
diff --git a/models/staging/exchange/_exchange__sources.yml b/models/staging/exchange/_exchange__sources.yml
new file mode 100644
index 0000000..d511863
--- /dev/null
+++ b/models/staging/exchange/_exchange__sources.yml
@@ -0,0 +1,619 @@
+version: 2
+
+sources:
+ - name: src_exchange_mysql
+ description: '{{ doc("src_exchange_mysql") }}'
+ database: |
+ {%- if target.name == "prod" -%} PROD_CIEF_RAW_DB
+ {%- else -%} DEV_CIEF_RAW_DB
+ {%- endif -%}
+ schema: EXCHANGE_MYSQL
+ loader: custom_python_code
+ loaded_at_field: _source_loaded_at
+ freshness:
+ warn_after: {count: 12, period: hour}
+ error_after: {count: 24, period: hour}
+ meta:
+ owner: "@yam"
+ model_maturity: prod
+ tags:
+ - exchange
+ - daily
+
+
+ tables:
+ - name: activity_log
+ description: Contains most table's activity log that CUD (created, updated, deleted) in www.exchange.cief-malaysia.com
+ columns:
+ - name: id
+ description: Primary key for 'activity_log'
+ tests:
+ - unique
+ - not_null
+
+ - name: description
+ description: system CUD activity list down
+ tests:
+ - accepted_values:
+ values: ['created', 'updated', 'deleted']
+
+ - name: subject_id
+ description: The exchange CUD table's id, need to refer subject_type column to know what table is CUD in the row
+ tests:
+ - not_null
+
+ - name: subject_type
+ description: What table that CUD using in the row
+ tests:
+ - not_null
+
+ - name: causer_id
+ description: Need to refer causer_type column to know who is CUD in the row. it is not well maintain there is some NULL
+
+ - name: causer_type
+ description: Who is CUD the row
+
+ - name: properties
+ description: Json data format that contain attributes of the activity, it is not well maintain, subject (Country, ServiceType, CurrencyRate. UserEmailVerification, File) have properties only
+
+ - name: created_at
+ description: Datetime that CUD activity occur
+ tests:
+ - not_null
+
+ - name: updated_at
+ description: Exactly same with created_at, redundant column
+
+ tests:
+ - dbt_expectations.expect_column_pair_values_to_be_equal:
+ column_A: updated_at
+ column_B: created_at
+
+
+ - name: addresses
+ description: Addresses that input when user register
+ columns:
+ - name: id
+ description: Primary key for 'addresses'
+ tests:
+ - unique
+ - not_null
+
+ - name: company_id
+ description: Foreign keys for 'companies'
+ tests:
+ - not_null
+
+ - name: country_id
+ description: Foreign keys for 'countries'
+ tests:
+ - not_null
+
+ - name: state_id
+ description: Foreign keys for 'states'
+ tests:
+ - not_null
+
+ - name: district_id
+ description: Foreign keys for 'districts'
+ tests:
+ - not_null
+
+ - name: postcode
+ description: The number/string that user input
+ tests:
+ - not_null
+ - dbt_expectations.expect_column_values_to_be_of_type:
+ column_type: integer
+
+ - name: street_one
+ description: The string that user input
+ tests:
+ - not_null
+
+ - name: street_two
+ description: The string that user input
+
+ - name: billing
+ description: If mark 1 means it is billing address, 0 means is not billing address, every company should have 1 billing address
+ tests:
+ - dbt_expectations.expect_column_max_to_be_between: #Every company should have a billing address
+ min_value: 1
+ max_value: 1
+ group_by: [company_id]
+ - dbt_expectations.expect_column_sum_to_be_between: #1 company should have 1 billing address only
+ min_value: 1
+ max_value: 1
+ group_by: [company_id]
+ row_condition: "deleted_at is null"
+
+ - name: deleted_at
+ description: System generate datetime that address was deleted
+
+ - name: created_at
+ description: System generate datetime that address was created
+ tests:
+ - not_null
+
+ - name: updated_at
+ description: System generate datetime that address was updated
+ tests:
+ - not_null
+
+
+ - name: announcement_segment
+ description: Superadmin / Operator guys will segment the announcment, but normally they will leave it default. 2 FK combine to becoem a relation table.
+ columns:
+ - name: announcement_id
+ description: Primary key for 'announcement_segment' & Foreign key for 'announcements'
+ tests:
+ - not_null
+
+ - name: segment_id
+ description: Primary key for 'announcement_segment' & Foreign key for 'segments'
+ tests:
+ - not_null
+
+ - name: created_at
+ description: Abandon column
+ tests:
+ - dbt_expectations.expect_column_values_to_be_null
+
+ - name: updated_at
+ description: Abandon column
+ tests:
+ - dbt_expectations.expect_column_values_to_be_null
+
+
+ - name: announcements
+ description: Announcements that show in our website which create by superadmin
+ columns:
+ - name: id
+ description: Primary key for 'announcements'
+ tests:
+ - unique
+ - not_null
+
+ - name: title
+ description: title of announcement, using string
+ tests:
+ - not_null
+
+ - name: description
+ description: description of announcement, using longtext, have html for break
+ tests:
+ - not_null
+
+ - name: starting_on
+ description: announcement starting datetime
+ tests:
+ - not_null
+
+ - name: ending_on
+ description: announcement ending datetime
+ tests:
+ - not_null
+
+ - name: is_all_day
+ description: column might not in use, system auto generate
+ tests:
+ - not_null
+ - accepted_values:
+ values: [0]
+
+ - name: duration
+ description: column might not in use, system auto generate
+ tests:
+ - not_null
+ - accepted_values:
+ values: [0]
+
+ - name: is_recurring
+ description: column might not in use, system auto generate
+ tests:
+ - not_null
+ - accepted_values:
+ values: [0]
+
+ - name: recurrence_pattern
+ description: column might not in use, system auto generate
+ tests:
+ - dbt_expectations.expect_column_values_to_be_null
+
+ - name: deleted_at
+ description: system generate datetime that announcment was deleted
+ tests:
+ - not_null
+
+ - name: created_at
+ description: system generate datetime that announcment was created
+ tests:
+ - not_null
+
+ - name: updated_at
+ description: system generate datetime that announcment was updated
+ tests:
+ - not_null
+
+
+ - name: bank_logs
+ description: Abandon table
+ config:
+ enabled: false
+
+
+ - name: banks
+ description: Table which store banks details.
+ columns:
+ - name: id
+ description: Primary key for 'banks'
+ tests:
+ - unique
+ - not_null
+
+ - name: company_id
+ description: Foreign keys for 'companies'
+ tests:
+ - not_null
+
+ - name: reference
+ description: A string which write by user/company for the bank transfer reference or other remind ussage
+
+ - name: holder_name
+ description: Bank holder name
+ tests:
+ - not_null
+
+ - name: account_no
+ description: Bank account number
+ tests:
+ - not_null
+
+
+
+
+ - name: booking_logs
+ description: Raw `bookings` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'booking_logs'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: bookings
+ description: Raw `bookings` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'bookings'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: companies
+ description: Raw `companies` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'companies'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: contacts
+ description: Raw `contacts` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'contacts'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: countries
+ description: Raw `countries` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'countries'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: currencies
+ description: Raw `currencies` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'currencies'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: currency_logs
+ description: Raw `currency_logs` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'currency_logs'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: currency_rate_logs
+ description: Raw `currency_rate_logs` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'currency_rate_logs'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: currency_rates
+ description: Raw `currency_rates` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'currency_rates'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: districts
+ description: Raw `districts` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'districts'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: documents
+ description: Raw `documents` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'documents'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: employees
+ description: Raw `employees` table from Snowflake
+ columns:
+ - name: company_id
+ description: Primary key for 'employees'
+
+
+ - name: failed_jobs
+ description: Raw `failed_jobs` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'failed_jobs'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: files
+ description: Raw `files` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'files'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: group_transactions
+ description: Raw `group_transactions` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'group_transactions'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: groups
+ description: Raw `groups` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'groups'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: jobs
+ description: Raw `jobs` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'jobs'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: migrations
+ description: Raw `migrations` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'migrations'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: model_has_roles
+ description: Raw `model_has_roles` table from Snowflake
+ columns:
+ - name: role_id
+ description: Primary key for 'model_has_roles'
+
+
+ - name: notifications
+ description: Raw `notifications` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'notifications'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: password_resets
+ description: Raw `password_resets` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'password_resets'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: permissions
+ description: Raw `permissions` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'permissions'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: role_has_permissions
+ description: Raw `role_has_permissions` table from Snowflake
+ columns:
+ - name: permission_id
+ description: Primary key for 'role_has_permissions'
+
+
+ - name: roles
+ description: Raw `roles` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'roles'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: segment_companies
+ description: Raw `segment_companies` table from Snowflake
+ columns:
+ - name: segment_id
+ description: Primary key for 'segment_companies'
+
+
+ - name: segment_constants
+ description: Raw `segment_constants` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'segment_constants'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: segments
+ description: Raw `segments` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'segments'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: service_types
+ description: Raw `service_types` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'service_types'
+ tests:
+ - unique
+ - not_null
+
+
+ description: Raw `states` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'states'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: transaction_detail
+ description: Raw `transaction_detail` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'transaction_detail'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: transaction_logs
+ description: Raw `transaction_logs` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'transaction_logs'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: transactions
+ description: Raw `transactions` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'transactions'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: user_email_verifications
+ description: Raw `user_email_verifications` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'user_email_verifications'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: users
+ description: Raw `users` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'users'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: wallet_logs
+ description: Raw `wallet_logs` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'wallet_logs'
+ tests:
+ - unique
+ - not_null
+
+
+ - name: wallets
+ description: Raw `wallets` table from Snowflake
+ columns:
+ - name: id
+ description: Primary key for 'wallets'
+ tests:
+ - unique
+ - not_null
\ No newline at end of file
diff --git a/models/staging/exchange/stg_exchange__activity_logs.sql b/models/staging/exchange/stg_exchange__activity_logs.sql
new file mode 100644
index 0000000..0040c52
--- /dev/null
+++ b/models/staging/exchange/stg_exchange__activity_logs.sql
@@ -0,0 +1,94 @@
+-- IMPORTS
+WITH activity_logs AS (
+ SELECT * FROM {{ source('src_exchange_mysql', 'activity_log') }}
+),
+
+
+-- LOGIC
+activity_logs_rename AS (
+
+ SELECT
+ activity_logs.id,
+ activity_logs.log_name,
+ activity_logs.description,
+ activity_logs.subject_id,
+ activity_logs.subject_type,
+ activity_logs.causer_id,
+ activity_logs.causer_type,
+ activity_logs.properties,
+ activity_logs.created_at,
+ activity_logs.updated_at,
+ current_timestamp() AS _dbt_ran_at
+
+ FROM activity_logs
+),
+
+
+-- FINAL
+final__base_exchange__activity_logs AS (
+
+ SELECT
+ id,
+ log_name,
+ description,
+ subject_id,
+ subject_type,
+ causer_id,
+ causer_type,
+ properties,
+ created_at,
+ updated_at,
+ _dbt_ran_at
+
+ FROM activity_logs_rename
+)
+
+SELECT * FROM final__base_exchange__activity_logs
+
+
+-- IMPORTS
+WITH activity_logs AS (
+ SELECT * FROM {{ ref('base_exchange__activity_logs') }}
+),
+
+
+-- LOGIC
+activity_logs_listout AS (
+
+ SELECT
+ activity_logs.id,
+ activity_logs.log_name,
+ activity_logs.description,
+ activity_logs.subject_id,
+ activity_logs.subject_type,
+ activity_logs.causer_id,
+ activity_logs.causer_type,
+ activity_logs.properties,
+ activity_logs.created_at,
+ activity_logs.updated_at,
+ CURRENT_TIMESTAMP() AS _dbt_ran_at
+
+ FROM activity_logs
+),
+
+
+-- FINAL
+final__stg_exchange__activity_logs AS (
+
+ SELECT
+ id,
+ log_name,
+ description,
+ subject_id,
+ subject_type,
+ causer_id,
+ causer_type,
+ properties,
+ created_at,
+ updated_at,
+ _dbt_ran_at
+
+ FROM activity_logs_listout
+)
+
+SELECT * FROM final__stg_exchange__activity_logs
\ No newline at end of file
diff --git a/models/exchange/staging/announcement_segments/stg_exchange__announcement_segments.sql b/models/staging/exchange/stg_exchange__announcement_segments.sql
similarity index 100%
rename from models/exchange/staging/announcement_segments/stg_exchange__announcement_segments.sql
rename to models/staging/exchange/stg_exchange__announcement_segments.sql
diff --git a/models/exchange/staging/announcements/stg_exchange__announcements.sql b/models/staging/exchange/stg_exchange__announcements.sql
similarity index 100%
rename from models/exchange/staging/announcements/stg_exchange__announcements.sql
rename to models/staging/exchange/stg_exchange__announcements.sql
diff --git a/models/exchange/staging/bank_logs/stg_exchange__bank_logs.sql b/models/staging/exchange/stg_exchange__bank_logs.sql
similarity index 100%
rename from models/exchange/staging/bank_logs/stg_exchange__bank_logs.sql
rename to models/staging/exchange/stg_exchange__bank_logs.sql
diff --git a/models/exchange/staging/banks/stg_exchange__banks.sql b/models/staging/exchange/stg_exchange__banks.sql
similarity index 100%
rename from models/exchange/staging/banks/stg_exchange__banks.sql
rename to models/staging/exchange/stg_exchange__banks.sql
diff --git a/models/exchange/staging/booking_logs/stg_exchange__booking_logs.sql b/models/staging/exchange/stg_exchange__booking_logs.sql
similarity index 100%
rename from models/exchange/staging/booking_logs/stg_exchange__booking_logs.sql
rename to models/staging/exchange/stg_exchange__booking_logs.sql
diff --git a/models/exchange/staging/bookings/stg_exchange__bookings.sql b/models/staging/exchange/stg_exchange__bookings.sql
similarity index 100%
rename from models/exchange/staging/bookings/stg_exchange__bookings.sql
rename to models/staging/exchange/stg_exchange__bookings.sql
diff --git a/models/exchange/staging/companies/stg_exchange__companies.sql b/models/staging/exchange/stg_exchange__companies.sql
similarity index 100%
rename from models/exchange/staging/companies/stg_exchange__companies.sql
rename to models/staging/exchange/stg_exchange__companies.sql
diff --git a/models/exchange/staging/companies_and_segments_relations/stg_exchange__companies_and_segments_relations.sql b/models/staging/exchange/stg_exchange__companies_and_segments_relations.sql
similarity index 100%
rename from models/exchange/staging/companies_and_segments_relations/stg_exchange__companies_and_segments_relations.sql
rename to models/staging/exchange/stg_exchange__companies_and_segments_relations.sql
diff --git a/models/exchange/staging/companies_and_users_relations/stg_exchange__companies_and_users_relations.sql b/models/staging/exchange/stg_exchange__companies_and_users_relations.sql
similarity index 100%
rename from models/exchange/staging/companies_and_users_relations/stg_exchange__companies_and_users_relations.sql
rename to models/staging/exchange/stg_exchange__companies_and_users_relations.sql
diff --git a/models/exchange/staging/company_addresses/stg_exchange__company_addresses.sql b/models/staging/exchange/stg_exchange__company_addresses.sql
similarity index 100%
rename from models/exchange/staging/company_addresses/stg_exchange__company_addresses.sql
rename to models/staging/exchange/stg_exchange__company_addresses.sql
diff --git a/models/exchange/staging/contacts/stg_exchange__contacts.sql b/models/staging/exchange/stg_exchange__contacts.sql
similarity index 100%
rename from models/exchange/staging/contacts/stg_exchange__contacts.sql
rename to models/staging/exchange/stg_exchange__contacts.sql
diff --git a/models/exchange/staging/countries/stg_exchange__countries.sql b/models/staging/exchange/stg_exchange__countries.sql
similarity index 100%
rename from models/exchange/staging/countries/stg_exchange__countries.sql
rename to models/staging/exchange/stg_exchange__countries.sql
diff --git a/models/exchange/staging/currencies/stg_exchange__currencies.sql b/models/staging/exchange/stg_exchange__currencies.sql
similarity index 100%
rename from models/exchange/staging/currencies/stg_exchange__currencies.sql
rename to models/staging/exchange/stg_exchange__currencies.sql
diff --git a/models/exchange/staging/currency_logs/stg_exchange__currency_logs.sql b/models/staging/exchange/stg_exchange__currency_logs.sql
similarity index 100%
rename from models/exchange/staging/currency_logs/stg_exchange__currency_logs.sql
rename to models/staging/exchange/stg_exchange__currency_logs.sql
diff --git a/models/exchange/staging/currency_rate_logs/stg_exchange__currency_rate_logs.sql b/models/staging/exchange/stg_exchange__currency_rate_logs.sql
similarity index 100%
rename from models/exchange/staging/currency_rate_logs/stg_exchange__currency_rate_logs.sql
rename to models/staging/exchange/stg_exchange__currency_rate_logs.sql
diff --git a/models/exchange/staging/current_currency_rates/stg_exchange__current_currency_rates.sql b/models/staging/exchange/stg_exchange__current_currency_rates.sql
similarity index 100%
rename from models/exchange/staging/current_currency_rates/stg_exchange__current_currency_rates.sql
rename to models/staging/exchange/stg_exchange__current_currency_rates.sql
diff --git a/models/exchange/staging/current_wallets/stg_exchange__current_wallets.sql b/models/staging/exchange/stg_exchange__current_wallets.sql
similarity index 100%
rename from models/exchange/staging/current_wallets/stg_exchange__current_wallets.sql
rename to models/staging/exchange/stg_exchange__current_wallets.sql
diff --git a/models/exchange/staging/districts/stg_exchange__districts.sql b/models/staging/exchange/stg_exchange__districts.sql
similarity index 100%
rename from models/exchange/staging/districts/stg_exchange__districts.sql
rename to models/staging/exchange/stg_exchange__districts.sql
diff --git a/models/exchange/staging/documents/stg_exchange__documents.sql b/models/staging/exchange/stg_exchange__documents.sql
similarity index 100%
rename from models/exchange/staging/documents/stg_exchange__documents.sql
rename to models/staging/exchange/stg_exchange__documents.sql
diff --git a/models/exchange/staging/failed_jobs/stg_exchange__failed_jobs.sql b/models/staging/exchange/stg_exchange__failed_jobs.sql
similarity index 100%
rename from models/exchange/staging/failed_jobs/stg_exchange__failed_jobs.sql
rename to models/staging/exchange/stg_exchange__failed_jobs.sql
diff --git a/models/exchange/staging/files/stg_exchange__files.sql b/models/staging/exchange/stg_exchange__files.sql
similarity index 100%
rename from models/exchange/staging/files/stg_exchange__files.sql
rename to models/staging/exchange/stg_exchange__files.sql
diff --git a/models/exchange/staging/group_costs/stg_exchange__group_costs.sql b/models/staging/exchange/stg_exchange__group_costs.sql
similarity index 100%
rename from models/exchange/staging/group_costs/stg_exchange__group_costs.sql
rename to models/staging/exchange/stg_exchange__group_costs.sql
diff --git a/models/exchange/staging/jobs/stg_exchange__jobs.sql b/models/staging/exchange/stg_exchange__jobs.sql
similarity index 100%
rename from models/exchange/staging/jobs/stg_exchange__jobs.sql
rename to models/staging/exchange/stg_exchange__jobs.sql
diff --git a/models/exchange/staging/migrations/stg_exchange__migrations.sql b/models/staging/exchange/stg_exchange__migrations.sql
similarity index 100%
rename from models/exchange/staging/migrations/stg_exchange__migrations.sql
rename to models/staging/exchange/stg_exchange__migrations.sql
diff --git a/models/exchange/staging/model_has_roles/stg_exchange__model_has_roles.sql b/models/staging/exchange/stg_exchange__model_has_roles.sql
similarity index 100%
rename from models/exchange/staging/model_has_roles/stg_exchange__model_has_roles.sql
rename to models/staging/exchange/stg_exchange__model_has_roles.sql
diff --git a/models/exchange/staging/notifications/stg_exchange__notifications.sql b/models/staging/exchange/stg_exchange__notifications.sql
similarity index 100%
rename from models/exchange/staging/notifications/stg_exchange__notifications.sql
rename to models/staging/exchange/stg_exchange__notifications.sql
diff --git a/models/exchange/staging/password_resets/stg_exchange__password_resets.sql b/models/staging/exchange/stg_exchange__password_resets.sql
similarity index 100%
rename from models/exchange/staging/password_resets/stg_exchange__password_resets.sql
rename to models/staging/exchange/stg_exchange__password_resets.sql
diff --git a/models/exchange/staging/permissions/stg_exchange__permissions.sql b/models/staging/exchange/stg_exchange__permissions.sql
similarity index 100%
rename from models/exchange/staging/permissions/stg_exchange__permissions.sql
rename to models/staging/exchange/stg_exchange__permissions.sql
diff --git a/models/exchange/staging/role_has_permissions/stg_exchange__role_has_permissions.sql b/models/staging/exchange/stg_exchange__role_has_permissions.sql
similarity index 100%
rename from models/exchange/staging/role_has_permissions/stg_exchange__role_has_permissions.sql
rename to models/staging/exchange/stg_exchange__role_has_permissions.sql
diff --git a/models/exchange/staging/roles/stg_exchange__roles.sql b/models/staging/exchange/stg_exchange__roles.sql
similarity index 100%
rename from models/exchange/staging/roles/stg_exchange__roles.sql
rename to models/staging/exchange/stg_exchange__roles.sql
diff --git a/models/exchange/staging/segment_constants/stg_exchange__segment_constants.sql b/models/staging/exchange/stg_exchange__segment_constants.sql
similarity index 100%
rename from models/exchange/staging/segment_constants/stg_exchange__segment_constants.sql
rename to models/staging/exchange/stg_exchange__segment_constants.sql
diff --git a/models/exchange/staging/segments/stg_exchange__segments.sql b/models/staging/exchange/stg_exchange__segments.sql
similarity index 100%
rename from models/exchange/staging/segments/stg_exchange__segments.sql
rename to models/staging/exchange/stg_exchange__segments.sql
diff --git a/models/exchange/staging/service_types/stg_exchange__service_types.sql b/models/staging/exchange/stg_exchange__service_types.sql
similarity index 100%
rename from models/exchange/staging/service_types/stg_exchange__service_types.sql
rename to models/staging/exchange/stg_exchange__service_types.sql
diff --git a/models/exchange/staging/states/stg_exchange__states.sql b/models/staging/exchange/stg_exchange__states.sql
similarity index 100%
rename from models/exchange/staging/states/stg_exchange__states.sql
rename to models/staging/exchange/stg_exchange__states.sql
diff --git a/models/exchange/staging/transaction_costs/stg_exchange__transaction_costs.sql b/models/staging/exchange/stg_exchange__transaction_costs.sql
similarity index 100%
rename from models/exchange/staging/transaction_costs/stg_exchange__transaction_costs.sql
rename to models/staging/exchange/stg_exchange__transaction_costs.sql
diff --git a/models/exchange/staging/transaction_costs_and_group_costs_relations/stg_exchange__transaction_costs_and_group_costs_relations.sql b/models/staging/exchange/stg_exchange__transaction_costs_and_group_costs_relations.sql
similarity index 100%
rename from models/exchange/staging/transaction_costs_and_group_costs_relations/stg_exchange__transaction_costs_and_group_costs_relations.sql
rename to models/staging/exchange/stg_exchange__transaction_costs_and_group_costs_relations.sql
diff --git a/models/exchange/staging/transaction_log_costs/stg_exchange__transaction_log_costs.sql b/models/staging/exchange/stg_exchange__transaction_log_costs.sql
similarity index 100%
rename from models/exchange/staging/transaction_log_costs/stg_exchange__transaction_log_costs.sql
rename to models/staging/exchange/stg_exchange__transaction_log_costs.sql
diff --git a/models/exchange/staging/transaction_log_orders/stg_exchange__transaction_log_orders.sql b/models/staging/exchange/stg_exchange__transaction_log_orders.sql
similarity index 100%
rename from models/exchange/staging/transaction_log_orders/stg_exchange__transaction_log_orders.sql
rename to models/staging/exchange/stg_exchange__transaction_log_orders.sql
diff --git a/models/exchange/staging/transaction_order_details/stg_exchange__transaction_order_details.sql b/models/staging/exchange/stg_exchange__transaction_order_details.sql
similarity index 100%
rename from models/exchange/staging/transaction_order_details/stg_exchange__transaction_order_details.sql
rename to models/staging/exchange/stg_exchange__transaction_order_details.sql
diff --git a/models/exchange/staging/transaction_orders/stg_exchange__transaction_orders.sql b/models/staging/exchange/stg_exchange__transaction_orders.sql
similarity index 100%
rename from models/exchange/staging/transaction_orders/stg_exchange__transaction_orders.sql
rename to models/staging/exchange/stg_exchange__transaction_orders.sql
diff --git a/models/exchange/staging/transaction_wallets/stg_exchange__transaction_wallets.sql b/models/staging/exchange/stg_exchange__transaction_wallets.sql
similarity index 100%
rename from models/exchange/staging/transaction_wallets/stg_exchange__transaction_wallets.sql
rename to models/staging/exchange/stg_exchange__transaction_wallets.sql
diff --git a/models/exchange/staging/transactions/stg_exchange__transactions.sql b/models/staging/exchange/stg_exchange__transactions.sql
similarity index 100%
rename from models/exchange/staging/transactions/stg_exchange__transactions.sql
rename to models/staging/exchange/stg_exchange__transactions.sql
diff --git a/models/exchange/staging/user_email_verifications/stg_exchange__user_email_verifications.sql b/models/staging/exchange/stg_exchange__user_email_verifications.sql
similarity index 100%
rename from models/exchange/staging/user_email_verifications/stg_exchange__user_email_verifications.sql
rename to models/staging/exchange/stg_exchange__user_email_verifications.sql
diff --git a/models/exchange/staging/users/stg_exchange__users.sql b/models/staging/exchange/stg_exchange__users.sql
similarity index 100%
rename from models/exchange/staging/users/stg_exchange__users.sql
rename to models/staging/exchange/stg_exchange__users.sql
diff --git a/models/staging/shipping/_shipping_sources.yml b/models/staging/shipping/_shipping_sources.yml
new file mode 100644
index 0000000..e69de29