UNComtrade
Data Product: United Nations Comtrade
Introduction
UNComtrade is responsible for getting global trade data for different countries. The trade data is regularly updated on a monthly basis. Normally, getting this data will require multiple days along with significant computational expenses. However this product contains and fetches the data upto the current month.
Nomenclature
- HS codes are 6 digit codes that are used as a standard for category of goods, they can be extended upto 10 digits with countries adding their own preferred codes at the end. They can also be aggregated to 4 or 2 digit codes. Here we are aggregating the trade data to 2 digit codes.
- The commodity codes are specified into various categories depending on the year in question. HS codes are used here to specify the categories. Major of these changes happen to 6 digit codes or 4 digit codes.
- H5 short for HS5 is the code standard for trades that took place between 2017-2022 and from 2022 onwards HS6 codes are being used, but still some countries are using HS5 codes.
- Exhaustive list for HS5 trade codes can be found at - Trade Statistics by Product (HS 6-digit)
HS6 Commodity Codes
| Commodity Code | Description |
|---|---|
| 01 | Animals; live |
| 02 | Meat and edible meat offal |
| 03 | Fish and crustaceans, molluscs and other aquatic invertebrates |
| 04 | Dairy produce; birds' eggs; natural honey; edible products of animal origin, not elsewhere specified or included |
| 05 | Animal originated products; not elsewhere specified or included |
| 06 | Trees and other plants, live; bulbs, roots and the like; cut flowers and ornamental foliage |
| 07 | Vegetables and certain roots and tubers; edible |
| 08 | Fruit and nuts, edible; peel of citrus fruit or melons |
| 09 | Coffee, tea, mate and spices |
| 10 | Cereals |
| 11 | Products of the milling industry; malt, starches, inulin, wheat gluten |
| 12 | Oil seeds and oleaginous fruits; miscellaneous grains, seeds and fruit, industrial or medicinal plants; straw and fodder |
| 13 | Lac; gums, resins and other vegetable saps and extracts |
| 14 | Vegetable plaiting materials; vegetable products not elsewhere specified or included |
| 15 | Animal, vegetable or microbial fats and oils and their cleavage products; prepared edible fats; animal or vegetable waxes |
| 16 | Meat, fish, crustaceans, molluscs or other aquatic invertebrates, or insects; preparations thereof |
| 17 | Sugars and sugar confectionery |
| 18 | Cocoa and cocoa preparations |
| 19 | Preparations of cereals, flour, starch or milk; pastrycooks' products |
| 20 | Preparations of vegetables, fruit, nuts or other parts of plants |
| 21 | Miscellaneous edible preparations |
| 22 | Beverages, spirits and vinegar |
| 23 | Food industries, residues and wastes thereof; prepared animal fodder |
| 24 | Tobacco and manufactured tobacco substitutes; products, whether or not containing nicotine, intended for inhalation without combustion; other nicotine containing products intended for the intake of nicotine into the human body |
| 25 | Salt; sulphur; earths, stone; plastering materials, lime and cement |
| 26 | Ores, slag and ash |
| 27 | Mineral fuels, mineral oils and products of their distillation; bituminous substances; mineral waxes |
| 28 | Inorganic chemicals; organic and inorganic compounds of precious metals; of rare earth metals, of radio-active elements and of isotopes |
| 29 | Organic chemicals |
| 30 | Pharmaceutical products |
| 31 | Fertilizers |
| 32 | Tanning or dyeing extracts; tannins and their derivatives; dyes, pigments and other colouring matter; paints, varnishes; putty, other mastics; inks |
| 33 | Essential oils and resinoids; perfumery, cosmetic or toilet preparations |
| 34 | Soap, organic surface-active agents; washing, lubricating, polishing or scouring preparations; artificial or prepared waxes, candles and similar articles, modelling pastes, dental waxes and dental preparations with a basis of plaster |
| 35 | Albuminoidal substances; modified starches; glues; enzymes |
| 36 | Explosives; pyrotechnic products; matches; pyrophoric alloys; certain combustible preparations |
| 37 | Photographic or cinematographic goods |
| 38 | Chemical products n.e.c. |
| 39 | Plastics and articles thereof |
| 40 | Rubber and articles thereof |
| 41 | Raw hides and skins (other than furskins) and leather |
| 42 | Articles of leather; saddlery and harness; travel goods, handbags and similar containers; articles of animal gut (other than silk-worm gut) |
| 43 | Furskins and artificial fur; manufactures thereof |
| 44 | Wood and articles of wood; wood charcoal |
| 45 | Cork and articles of cork |
| 46 | Manufactures of straw, esparto or other plaiting materials; basketware and wickerwork |
| 47 | Pulp of wood or other fibrous cellulosic material; recovered (waste and scrap) paper or paperboard |
| 48 | Paper and paperboard; articles of paper pulp, of paper or paperboard |
| 49 | Printed books, newspapers, pictures and other products of the printing industry; manuscripts, typescripts and plans |
| 50 | Silk |
| 51 | Wool, fine or coarse animal hair; horsehair yarn and woven fabric |
| 52 | Cotton |
| 53 | Vegetable textile fibres; paper yarn and woven fabrics of paper yarn |
| 54 | Man-made filaments; strip and the like of man-made textile materials |
| 55 | Man-made staple fibres |
| 56 | Wadding, felt and nonwovens, special yarns; twine, cordage, ropes and cables and articles thereof |
| 57 | Carpets and other textile floor coverings |
| 58 | Fabrics; special woven fabrics, tufted textile fabrics, lace, tapestries, trimmings, embroidery |
| 59 | Textile fabrics; impregnated, coated, covered or laminated; textile articles of a kind suitable for industrial use |
| 60 | Fabrics; knitted or crocheted |
| 61 | Apparel and clothing accessories; knitted or crocheted |
| 62 | Apparel and clothing accessories; not knitted or crocheted |
| 63 | Textiles, made up articles; sets; worn clothing and worn textile articles; rags |
| 64 | Footwear; gaiters and the like; parts of such articles |
| 65 | Headgear and parts thereof |
| 66 | Umbrellas, sun umbrellas, walking-sticks, seat sticks, whips, riding crops; and parts thereof |
| 67 | Feathers and down, prepared; and articles made of feather or of down; artificial flowers; articles of human hair |
| 68 | Stone, plaster, cement, asbestos, mica or similar materials; articles thereof |
| 69 | Ceramic products |
| 70 | Glass and glassware |
| 71 | Natural, cultured pearls; precious, semi-precious stones; precious metals, metals clad with precious metal, and articles thereof; imitation jewellery; coin |
| 72 | Iron and steel |
| 73 | Iron or steel articles |
| 74 | Copper and articles thereof |
| 75 | Nickel and articles thereof |
| 76 | Aluminium and articles thereof |
| 78 | Lead and articles thereof |
| 79 | Zinc and articles thereof |
| 80 | Tin; articles thereof |
| 81 | Metals; n.e.c., cermets and articles thereof |
| 82 | Tools, implements, cutlery, spoons and forks, of base metal; parts thereof, of base metal |
| 83 | Metal; miscellaneous products of base metal |
| 84 | Machinery and mechanical appliances, boilers, nuclear reactors; parts thereof |
| 85 | Electrical machinery and equipment and parts thereof; sound recorders and reproducers; television image and sound recorders and reproducers, parts and accessories of such articles |
| 86 | Railway, tramway locomotives, rolling-stock and parts thereof; railway or tramway track fixtures and fittings and parts thereof; mechanical (including electro-mechanical) traffic signalling equipment of all kinds |
| 87 | Vehicles; other than railway or tramway rolling stock, and parts and accessories thereof |
| 88 | Aircraft, spacecraft, and parts thereof |
| 89 | Ships, boats and floating structures |
| 90 | Optical, photographic, cinematographic, measuring, checking, medical or surgical instruments and apparatus; parts and accessories |
| 91 | Clocks and watches and parts thereof |
| 92 | Musical instruments; parts and accessories of such articles |
| 93 | Arms and ammunition; parts and accessories thereof |
| 94 | Furniture; bedding, mattresses, mattress supports, cushions and similar stuffed furnishings; lamps and lighting fittings, n.e.c.; illuminated signs, illuminated name-plates and the like; prefabricated buildings |
| 95 | Toys, games and sports requisites; parts and accessories thereof |
| 96 | Miscellaneous manufactured articles |
| 97 | Works of art; collectors' pieces and antiques |
| 99 | Commodities not specified according to kind |
Modules
Scraping
Scraping module gets the data from the UN Comtrade database for the last three years for partner countries storedin our database and thoroughly filters and saves relevant data using: https://comtrade.un.org/data/
Cleaning
Cleaning module is responsible for erasing duplicate values.
Standardization
Used to standardize the data according to Taiyo standards. Introduces new fields and improves the visibility ofpreviously inserted fields. Adds partner_country_code, region_code, region, domain, subdomain Also, standardization gets old data from the database and appends it to the new data At each step, data is stored into s3 buckets for better workflows
Metadata
Metadata module is used for attaching a ts_ref_id to each reporter country In the end all data from each and every module is stored as a csv file in our s3 bucket
Data Format
Metadata Attributes
| Attributes | Descriptions |
|---|---|
| ts_ref_id | unique value to make querying easier |
| value | Trade value in dollars |
| timestamp | Time of trade as reported by the reporter country |
| partner | country with which the reporter has reported the trade |
| reporter_country | country which reports a trade event, the flow(import/export) is relative to the reporter |
| country_code | 3 digit ISO code given to the country by the world bank |
| region_code | Region code determined by the World Bank. |
| region | Region name as designated by the World Bank. |
| domain | Field of data determined by Taiyo standards |
| subdomain | The type in which the data fits in |
| sample_frequency | Frequency at which data is collected |
| commodity | The name/category of goods being traded |
| commodity code | code specific to the goods being traded |
| flow | Represent the flow of trade(import/export) |
| url | address from which the data was imported |
| name | Name of the source |
| description | a short description of the trade, i.e which country is importing/ exporting |
| source | From where the data was fetched |
| time_of_sampling | time at which the data was inserted |
| date_of_sampling | date at which the data was inserted |
## Data Flow
Taiyo Data Format
| Entity | UNComtrade |
|---|---|
| Frequency | Monthly |
| Updated On | 02-06-2022 UTC 06:42:59 AM |
| Coverage | 150+ countries |
| Uncertainties | Some countries don’t report their trades so they are approximated using their partner countries |
Scope of Improvement
- The category codes can be increased to 4/6 digit codes from 2 digit codes.
Useful Links
- https://comtrade.un.org/data/
- https://countrycode.org/
- https://comtrade.un.org/data/cache/classificationH5.json
- https://www.trade.gov/harmonized-system-hs-codes
- https://unstats.un.org/unsd/trade/classifications/correspondence-tables.asp
- https://wits.worldbank.org/trade/country-byhs6product.aspx?lang=en