cief/data-management
1
0
Fork 0
mirror of https://gitlab.com/cief-data/data-management.git synced 2026-08-19 20:33:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

Readme.md update

Browse Source
This commit is contained in:
Yam ZhengLim
2022-08-22 19:39:01 +08:00
parent 4eeeeaa478
commit 1e830fa496
1 changed files with 31 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+31 -1
View File
@@ -8,4 +8,34 @@
## ✨ Code-base structure
### How To Run CMD:
##### python run_extract_load.py
#### First Run
#### __________________________________
###### pip install -r requirements.txt
#### __________________________________
#### Run the extract load code
###### python run_extract_load.py
```
data-management
│
├─ .gitignore # Ignore the file to push into gitlab
│
├─ config # Folder store all the config file to use in the code
│ ├─ CLOUD-STORAGE-ADMIN-CREDENTIAL.json # Get from GCP Service Account cloud-storage-admin-service-account
│ ├─ EXCHANGE_CONFIG.json # Manually config the MySQL ID, Password, etc...
│ └─ EXCHANGE_EXTRACT_SQL.json # SQL use in MYSQL to extract the data we need to save in temp's csv
│
├─ exchange # Exchange web application
│ ├─ extract # All extract needed code is in here
│ │ ├─ exchange_extract_function.py # Exchange extract code function to import in run_extract_load.py
│ │ └─ exchange_temp_extract_csv # Temp folder to store all the csv extract from MYSQL
│ │
│ └─ load # All load needed code is in here
│ └─ exchange_load_function.py # Exchange load code function to import in run_extract_load.py
│
├─ general_function.py # All customize general code function to import in run_extract_load.py
├─ README.md # Project guide
├─ requirements.txt # Install needed library
└─ run_extract_load.py # Main code to run the extract load process
```