cief/data-management
1
0
Fork 0
mirror of https://gitlab.com/cief-data/data-management.git synced 2026-08-19 12:23:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

Commit Graph

  • 6ef680b84b debug typo main Yam ZhengLim 2024-05-20 17:38:32 +08:00
  • 51ce8d6ee7 Comment CRISP as it not manage already Yam ZhengLim 2024-04-07 09:45:09 +00:00
  • d72e9d4873 Update README.md Yam ZhengLim 2024-04-07 09:40:04 +00:00
  • ee0d22ed92 Update EXCHANGE_EXTRACT_SQL.json Yam ZhengLim 2024-04-07 09:28:59 +00:00
  • 25aef150ee Update SHIPPING_EXTRACT_SQL.json Yam ZhengLim 2024-04-07 09:26:15 +00:00
  • ada8e5150b Update EXCHANGE_EXTRACT_SQL.json Yam ZhengLim 2024-04-07 09:11:02 +00:00
  • 25efe05695 Add c2m into extractload script Yam ZhengLim 2024-01-27 18:43:38 +08:00
  • 5d19b5a567 Debug Headless Yam ZhengLim 2024-01-27 18:28:09 +08:00
  • 1f86520c77 Healess = True Yam ZhengLim 2024-01-27 18:26:07 +08:00
  • e3cbdf05ae Test remove headless Yam ZhengLim 2024-01-27 18:23:30 +08:00
  • a64d161662 Debug Headless Yam ZhengLim 2024-01-27 18:20:31 +08:00
  • da5162952f Debug Headless Yam ZhengLim 2024-01-27 18:19:32 +08:00
  • 4ff602f2c6 C2M Scrape add Yam ZhengLim 2024-01-27 14:46:57 +08:00
  • e53a9817a6 Update EXCHANGE & SHIPPING raw data Yam ZhengLim 2023-07-05 13:17:45 +08:00
  • 9762e3721e SHIPPING Add source_loaded_datetime Yam ZhengLim 2023-03-31 11:51:29 +08:00
  • ac29d06220 Exchange add source loaded datetime Yam ZhengLim 2023-03-31 11:37:57 +08:00
  • 164e72d1d3 Crisp Extract Load add_source_loaded_datetime Yam ZhengLim 2023-02-23 22:31:41 +08:00
  • 596a522f08 Open back all function Yam ZhengLim 2023-02-22 01:53:54 +08:00
  • 1c5b65e79b Update debug Yam ZhengLim 2023-02-22 01:48:26 +08:00
  • a133311917 Add more print indicate Yam ZhengLim 2023-02-22 00:51:39 +08:00
  • 04692f644e Add print indecate Yam ZhengLim 2023-02-22 00:50:26 +08:00
  • d076644241 Unmask hard code masking Yam ZhengLim 2023-02-22 00:39:10 +08:00
  • 42aeedc9df Debug Crisp Yam ZhengLim 2022-12-28 11:44:32 +08:00
  • b94835013a Debug ShippingSQL Yam ZhengLim 2022-12-28 11:40:57 +08:00
  • db7c97aa57 Update naming for SNOWFLAKE Yam ZhengLim 2022-12-28 00:11:48 +08:00
  • 827499a309 Debug CRISP naming Yam ZhengLim 2022-12-26 16:50:22 +08:00
  • d28dbff76a Debug timestamp Yam ZhengLim 2022-12-26 15:55:11 +08:00
  • 951e5e970e Add timestamp Yam ZhengLim 2022-12-23 16:39:33 +08:00
  • 8439a617f1 update crisp function Yam ZhengLim 2022-12-22 15:38:51 +08:00
  • a3501cba2f Open all EL function Yam ZhengLim 2022-12-19 15:30:21 +08:00
  • 263177c717 Pause for CRISP Yam ZhengLim 2022-12-19 15:26:50 +08:00
  • cc7cea07a5 Add CRISP & ExchangeLog Data Yam ZhengLim 2022-12-19 15:19:37 +08:00
  • 5b3406e816 Debug CRISP Yam ZhengLim 2022-12-08 00:22:59 +08:00
  • e1de2e3668 Update API Key CRISP Yam ZhengLim 2022-12-06 23:06:23 +08:00
  • 7adb86fab9 Update documentation Yam ZhengLim 2022-12-06 22:34:32 +08:00
  • f60b4f25a0 Debug Yam ZhengLim 2022-12-06 22:19:21 +08:00
  • 828a29d052 Crisp ExtractLoad user only Yam ZhengLim 2022-12-06 22:18:48 +08:00
  • 8f2b2b1896 Debug SQL Yam ZhengLim 2022-11-30 18:10:06 +08:00
  • 14f9a9929a Debug Yam ZhengLim 2022-11-30 01:13:09 +08:00
  • cc8f58e518 Debug SQL Yam ZhengLim 2022-11-30 01:12:42 +08:00
  • 51cf7fe84f requirement Yam ZhengLim 2022-11-30 01:05:38 +08:00
  • 0def9a4c54 ETLT and CRISP Yam ZhengLim 2022-11-29 20:23:27 +08:00
  • 22ed4e23e9 Debug SQL Yam ZhengLim 2022-11-14 00:45:39 +08:00
  • 014bca9416 Remove Job SQL Yam ZhengLim 2022-11-14 00:08:09 +08:00
  • 8b78ea8f10 Full run Yam ZhengLim 2022-11-13 23:47:03 +08:00
  • 3ca96ae91a Remark SQL debug Yam ZhengLim 2022-11-13 23:43:39 +08:00
  • 3f2785279f Test Shipping Yam ZhengLim 2022-11-13 21:07:05 +08:00
  • e940b729e9 Add shipping ExtractLoad Yam ZhengLim 2022-11-13 21:06:35 +08:00
  • 3f25d3debc Change X1 to Shipping Yam ZhengLim 2022-11-13 18:00:42 +08:00
  • 4d3b345935 Debug SQL Yam ZhengLim 2022-11-12 13:13:09 +08:00
  • 3ece93a1aa test X1 Yam ZhengLim 2022-11-12 12:55:30 +08:00
  • b29f1e3baf Add sleep Yam ZhengLim 2022-11-12 12:53:20 +08:00
  • cbdbfca0c8 Update code Yam ZhengLim 2022-11-12 12:47:22 +08:00
  • d9b7302eff Change database for X1 Yam ZhengLim 2022-11-12 12:39:02 +08:00
  • acff209f06 Add X1 Yam ZhengLim 2022-11-12 12:30:42 +08:00
  • 12eed9fad4 Remove source loaded at Yam ZhengLim 2022-11-04 22:20:18 +08:00
  • 27e470f444 Debug Yam ZhengLim 2022-11-04 22:04:45 +08:00
  • 47c84604bd remove source_updated_at Yam ZhengLim 2022-11-04 21:43:26 +08:00
  • 50594b476b now() to null Yam ZhengLim 2022-11-04 21:28:44 +08:00
  • cb5d6afe7c Update unwanted file Yam ZhengLim 2022-10-29 10:46:24 +08:00
  • f22b914f59 Update SQL Yam ZhengLim 2022-10-29 10:44:57 +08:00
  • d4d17efa3d Debug EL Yam ZhengLim 2022-10-29 01:31:08 +08:00
  • 5d30c6c505 Fully ELT Yam ZhengLim 2022-10-28 15:40:00 +08:00
  • cb08e5b561 Update fully ELT Yam ZhengLim 2022-10-28 15:39:09 +08:00
  • 743279b868 Add Currency_Rate_Logs SQL Yam ZhengLim 2022-10-21 18:08:50 +08:00
  • bfd5f83435 debug typo Yam ZhengLim 2022-10-14 22:38:46 +08:00
  • 84b76f2d72 email change to email_hash_id Yam ZhengLim 2022-10-14 22:33:31 +08:00
  • ea76f925f1 Upadate README Yam ZhengLim 2022-10-09 17:43:11 +08:00
  • 173a56be1c Wallet Code mask Yam ZhengLim 2022-10-09 17:42:22 +08:00
  • 97b045b268 Backslash debug Yam ZhengLim 2022-10-02 20:54:37 +08:00
  • 1eb5c442ba Debug cont Yam ZhengLim 2022-10-02 20:46:21 +08:00
  • fe73874095 Debug snowflake Yam ZhengLim 2022-10-02 20:40:47 +08:00
  • 1aec3161d2 Debug snowflake Yam ZhengLim 2022-10-02 20:35:59 +08:00
  • 02789f71d8 Debug Yam ZhengLim 2022-10-02 14:42:36 +08:00
  • a844b012db SQL Update Yam ZhengLim 2022-10-02 14:27:39 +08:00
  • 7dcc402dab SQL transform Yam ZhengLim 2022-10-02 11:02:05 +08:00
  • c56ac245e2 newline debig Yam ZhengLim 2022-10-02 10:38:07 +08:00
  • 4b67fcb215 quote minimal Yam ZhengLim 2022-10-01 12:46:47 +08:00
  • e938412405 Quoting Yam ZhengLim 2022-10-01 12:41:54 +08:00
  • 81c1240624 Remove | Yam ZhengLim 2022-10-01 12:34:42 +08:00
  • 45dff350f5 Remove | Yam ZhengLim 2022-10-01 12:34:19 +08:00
  • 5b18ac0ce6 test delimiter and quotechar Yam ZhengLim 2022-10-01 12:25:41 +08:00
  • 6c8338ad5d delimiter |_| Yam ZhengLim 2022-10-01 12:24:20 +08:00
  • 66ebfb6474 delete quote Yam ZhengLim 2022-10-01 12:21:05 +08:00
  • b949b7d496 remove boto3 Yam ZhengLim 2022-10-01 12:10:59 +08:00
  • 2738b8bd82 enclose debug Yam ZhengLim 2022-10-01 12:08:19 +08:00
  • bf3f0743d7 Encoding chinese Yam ZhengLim 2022-09-30 22:23:14 +08:00
  • 7c9adbe5f6 Merge branch 'main' of gitlab.com:CIEFWorldwideSdnBhd/data-management Yam ZhengLim 2022-09-30 22:15:04 +08:00
  • 8ee3ca915b debug mysql connector Yam ZhengLim 2022-09-30 22:14:42 +08:00
  • fd92c3d7ac DOWNGRADE mysql-connector-python to 8.0.29 Yam ZhengLim 2022-09-30 14:11:17 +00:00
  • a12b365003 masking update Yam ZhengLim 2022-09-29 23:40:19 +08:00
  • 7df890b90a Readme update Yam ZhengLim 2022-09-26 19:58:05 +08:00
  • 0ef5784893 Permenant remove linux Yam ZhengLim 2022-09-26 19:51:26 +08:00
  • 92affec2c5 Output bug fix Yam ZhengLim 2022-09-26 19:50:32 +08:00
  • 2042ebdf7b Output config Yam ZhengLim 2022-09-26 19:48:15 +08:00
  • f530631229 Change SQL Yam ZhengLim 2022-09-26 19:46:16 +08:00
  • 13d7d0545e add function into gen_func Yam ZhengLim 2022-09-26 19:42:02 +08:00
  • 7d60227b89 Change readme & Needed SQL Yam ZhengLim 2022-09-26 19:35:28 +08:00
  • b68ae33fd1 Remove file Yam ZhengLim 2022-09-26 19:09:13 +08:00
  • cd06db9aeb Readme update Yam ZhengLim 2022-09-26 19:03:10 +08:00