AFDB - African Development Bank
Introduction
The overarching objective of the African Development Bank (AfDB) Group is to spur sustainable economic development and social progress in its regional member countries (RMCs), thus contributing to poverty reduction.
The Bank Group achieves this objective by:
- mobilizing and allocating resources for investment in RMCs; and
- providing policy advice and technical assistance to support development efforts.
In 2015, all multilateral development institutions agreed on the same set of objectives, called the Sustainable Development Goals.
Source: AFDB Projects
Tags: Multilateral, Government Announcement, Public Procurement, Infrastructure, Construction
Modules
Scrapping:
Scrapper uses the AFDB Projects page to get a list of all projects listed there. It also scrapes the geospatial data. Scrapper goes through all the pages for each project, collecting all the data and stores it in a single CSV. This CSV is stored in the bucket.
Cleaning:
Currency conversion, handling null values, removing duplicate entries, timestamp formatting, and sector & subsector cleaning is done.
Standardization:
Additional information like sample frequency, units, source and description are included in the metadata. Predefined sectors and subsectors are added in this step.
MetaData:
Metadata contains timestamp(approval date) range, sectors, subsectors and country codes. And is also stored in the s3 bucket.
Ingest:
Metadata and project data are ingested into an Elastic Search cluster. Index created based on fields - sector, subsector, map coordinates.
Metadata
Metadata Attributes
| Attributes | Descriptions |
|---|---|
| original_id | Original ID used by source |
| name | Name of the Project represented in the data point |
| url | Link to project page |
| description | Project description |
| status | Status of the project as identified by AFDB |
| development_objective | The project development objective |
| benefits | The benefits of the proposed project and its beneficiaries |
| rational | The rationale for the project |
| source_of_financing | Financing sources working with AFDB |
| sector | Sector based on AFDB categorization |
| country_name | Place of performance - Country Name |
| approval_date | Approval date |
| budget | Budget of the project calculated using financial details of the project |
| implementing_agency | Implementing agency of the project |
| currency | Currency |
| task_manager | Task Manager for the project at AFDB |
| location | Place of performance - Location |
| closing_date | Closing Date |
| environmental_category | Environmental category as identified by AFDB |
| last_update_date | Last Update Date |
| project_or_tender | Label for Project or Tender |
| aug_id | Unique ID generated for identification of the project |
| approval_year | Approval Year |
| subsector | Subector based on AFDB categorization |
| country_code | Country Code |
| region_name | Region Name |
| region_code | Region Code |
| total_investment | Total Investment in the project calculated using financial details of the project |
| source | Source |
| identified_sector | Identified Sector |
| identified_subsector | Identified Subsector |
| identified_sector_subsector_tuple | Identified Subsector |
| map_coordinates | Map Coordinates in decimal degrees |
| identified_status | Identified Status |
Data Flow
The above data pipeline runs on Argo and it will be executed on a daily frequency.
DAGs:
- AFDB: Total No of DAGs file is 1
Scope for Improvement
The following can be improved in the next version of the data product:
- In future, we might want to improve it to only scrap the data that we don’t already have.
Useful Links
- https://www.afdb.org/en/projects-and-operations
- https://www.afdb.org/en/about/mission-strategy