Global Trade Alert
Introduction
Global Trade Alert provides timely information on state interventions taken since November 2008 that are likely to affect foreign commerce. It includes state interventions affecting trade in goods and services, foreign investment and labor force migration.
Global Trade Alert has 3 indicators whose data is stored in the database along with the information related to trade alert. Global Trade Alert is event-based data which is updated sporadically on the website depending on global trade policies.
Source: Global Trade Alert
Tags: Time-series, Risk, Daily
Modules
Scrapping:
Scrapper gets data from the global trade alert for all the latest alert that has been added to the website and stored them in a raw format.
Cleaning:
In Cleaning step, data field name correction and cleaning take place. Timestamp format is changed to standard timeseries timestamp and added to the data.
Standardization:
Region, region code, country code and source are added to the data. Sector tags, domain and subdomain details are added in this step.
Geocoder:
In Geocoder step, coordinates are added to the metadata for the countries in the global trade alert data. Geocoder library is used for getting coordinates.
MetaData:
Create and append ts_ref_id to the data to create timeseries and then extract metadata from the series and save data to the s3 bucket.
Ingest:
Metadata and timeseries data are ingested in the mongoDB and latest timestamp id (mongoDB id for latest timestamp) is appended to metadata for decreasing search for latest data point.
Data Format
Timeseries Attributes
Attributes | Descriptions |
---|---|
ts_ref_id | Id used to connect timeseries data to the metadata. |
value | Timeseries information stored for global trade alert indicator data. |
timestamp | standard timestamp used for the timeseries. |
### Metadata Attributes | |
Attributes | Descriptions |
- | - |
ts_ref_id | Id used to connect metadata to the timeseries |
implementing_country | Country that is implementing the policy |
identifier | There are three identifiers in the global trade harmful: alert that has negative impact on countries neutral: alert that has neutral impact on countries liberalizing: alert that has positive impact on countries |
affected_countries | country that has been impacted due to trade policy |
intervention_id | id for interventions in the trade policy |
title | name of the alert |
date_of_sampling | date on which data is collected |
timezone | Timezone for the time and date |
time_of_sampling | time of data collection |
country | implementing country given on the website |
country_code | ISO3 alpha-3 code for the country |
map_coordinates | Latitude and Longitude of the country |
region | region for a country according to World Bank Standards. |
region_code | region code for a region according to World Bank Standards |
source | website from which data is taken from |
source_url | urls of the sources from which global alert is taken |
domain | Predefined domain by Taiyo |
subdomain | Predefined subdomain by Taiyo. |
affected_sector | HS (version 2012) at the 6-digit levelof the sector that are affected |
affected_product | CPC (version 2.1) at the 3-digit levelof the product that are affected |
sector_tags | full name of the sector that are affected |
url | urls of alert on the global trade alert |
latest_timestamp_id | mongoDB id for latest timestamp in the timeseries. |
Data Flow
The above data pipeline runs on Argo and it will be executed on a periodic frequency.
Taiyo Data Format
Entity | Global Trade Alerts |
---|---|
Frequency | Event Based |
Updated On | 24-04-2022 UTC 08:20:00 AM |
Coverage | 3 indicators data is collected for more than 100 countries. |
Uncertainties | Data is collected with utmost precision but alert data for some instance may not be stored in database due to data corruption at source. |
Scope for Improvement
Following can be improved in the next version of the data product:
· Performance of the scrapper can be improved and data can be stored more efficiently