{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Ingest Digital Obstacle Data\n", "\n", "We're going to get a dataset of aeronautical obstructions call the Digital Obstruction File. This is maintained by the US Federal Aviation Administratrion (FAA) and downloadable from [the FAA's DOF site](https://aeronav.faa.gov/Obst_Data/DAILY_DOF_CSV.ZIP). Take a look at [the FAQs](https://www.faa.gov/air_traffic/flight_info/aeronav/obst_data/doffaqs/).\n", "\n", "## Download and check the data" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "--2020-02-23 15:56:01-- https://aeronav.faa.gov/Obst_Data/DAILY_DOF_CSV.zip\n", "Resolving aeronav.faa.gov (aeronav.faa.gov)... 155.178.201.160\n", "Connecting to aeronav.faa.gov (aeronav.faa.gov)|155.178.201.160|:443... connected.\n", "HTTP request sent, awaiting response... 200 OK\n", "Length: 15056154 (14M) [application/x-zip-compressed]\n", "Saving to: ‘DAILY_DOF_CSV.zip’\n", "\n", "DAILY_DOF_CSV.zip 100%[===================>] 14.36M 1003KB/s in 16s \n", "\n", "2020-02-23 15:56:17 (943 KB/s) - ‘DAILY_DOF_CSV.zip’ saved [15056154/15056154]\n", "\n", "Archive: DAILY_DOF_CSV.ZIP\n", " inflating: DOF.CSV \n" ] } ], "source": [ "!rm DAILY_DOF_CSV.zip DOF.CSV\n", "!wget https://aeronav.faa.gov/Obst_Data/DAILY_DOF_CSV.zip\n", "!unzip -o DAILY_DOF_CSV.ZIP" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can preview the data. Not using an encoding, or using something like `utf-8` chokes. Following [a tip about dealing with data that was exported from Excel](https://srinujammu.wordpress.com/2018/03/01/how-to-avoid-unicodedecodeerror-while-loading-data-into-dataset-in-python/), the use of `cp1252` seems to fix things." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(483167, 19)\n" ] }, { "data": { "text/html": [ "
\n", " | OAS | \n", "VERIFIED STATUS | \n", "COUNTRY | \n", "STATE | \n", "CITY | \n", "LATDEC | \n", "LONDEC | \n", "DMSLAT | \n", "DMSLON | \n", "TYPE | \n", "QUANTITY | \n", "AGL | \n", "AMSL | \n", "LIGHTING | \n", "ACCURACY | \n", "MARKING | \n", "FAA STUDY | \n", "ACTION | \n", "JDATE | \n", "geometry | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | \n", "01-001307 | \n", "O | \n", "US | \n", "AL | \n", "DAUPHIN ISLAND | \n", "30.179167 | \n", "-88.077500 | \n", "30 10 45.00N | \n", "088 04 39.00W | \n", "RIG | \n", "1 | \n", "236 | \n", "236 | \n", "R | \n", "5D | \n", "M | \n", "1990ASO01578OE | \n", "C | \n", "2014138 | \n", "POINT (-88.07750 30.17917) | \n", "
1 | \n", "01-001459 | \n", "O | \n", "US | \n", "AL | \n", "DAUPHIN ISLAND | \n", "30.188889 | \n", "-88.120834 | \n", "30 11 20.00N | \n", "088 07 15.00W | \n", "RIG | \n", "1 | \n", "240 | \n", "241 | \n", "R | \n", "5D | \n", "M | \n", "1992ASO02229OE | \n", "C | \n", "2014138 | \n", "POINT (-88.12083 30.18889) | \n", "
2 | \n", "01-001472 | \n", "O | \n", "US | \n", "AL | \n", "FORT MORGAN | \n", "30.188889 | \n", "-87.952778 | \n", "30 11 20.00N | \n", "087 57 10.00W | \n", "STACK | \n", "1 | \n", "193 | \n", "193 | \n", "R | \n", "5D | \n", "M | \n", "1992ASO02230OE | \n", "C | \n", "2014138 | \n", "POINT (-87.95278 30.18889) | \n", "
3 | \n", "01-002558 | \n", "O | \n", "US | \n", "AL | \n", "GULF SHORES | \n", "30.230278 | \n", "-87.875000 | \n", "30 13 49.00N | \n", "087 52 30.00W | \n", "BLDG | \n", "1 | \n", "223 | \n", "242 | \n", "R | \n", "5D | \n", "N | \n", "1999ASO03256OE | \n", "C | \n", "2004005 | \n", "POINT (-87.87500 30.23028) | \n", "
4 | \n", "01-002567 | \n", "O | \n", "US | \n", "AL | \n", "GULF SHORES | \n", "30.230278 | \n", "-87.873612 | \n", "30 13 49.00N | \n", "087 52 25.00W | \n", "BLDG | \n", "1 | \n", "223 | \n", "242 | \n", "R | \n", "5D | \n", "N | \n", "2000ASO03180OE | \n", "C | \n", "2004005 | \n", "POINT (-87.87361 30.23028) | \n", "
\n", " | scalerank | \n", "labelrank | \n", "sovereignt | \n", "sov_a3 | \n", "adm0_dif | \n", "level | \n", "type | \n", "admin | \n", "adm0_a3 | \n", "geou_dif | \n", "... | \n", "region_un | \n", "subregion | \n", "region_wb | \n", "name_len | \n", "long_len | \n", "abbrev_len | \n", "tiny | \n", "homepart | \n", "featureclass | \n", "geometry | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | \n", "1 | \n", "3 | \n", "Afghanistan | \n", "AFG | \n", "0 | \n", "2 | \n", "Sovereign country | \n", "Afghanistan | \n", "AFG | \n", "0 | \n", "... | \n", "Asia | \n", "Southern Asia | \n", "South Asia | \n", "11 | \n", "11 | \n", "4 | \n", "-99 | \n", "1 | \n", "Admin-0 country | \n", "POLYGON ((61.21082 35.65007, 62.23065 35.27066... | \n", "
1 | \n", "1 | \n", "3 | \n", "Angola | \n", "AGO | \n", "0 | \n", "2 | \n", "Sovereign country | \n", "Angola | \n", "AGO | \n", "0 | \n", "... | \n", "Africa | \n", "Middle Africa | \n", "Sub-Saharan Africa | \n", "6 | \n", "6 | \n", "4 | \n", "-99 | \n", "1 | \n", "Admin-0 country | \n", "MULTIPOLYGON (((16.32653 -5.87747, 16.57318 -6... | \n", "
2 | \n", "1 | \n", "6 | \n", "Albania | \n", "ALB | \n", "0 | \n", "2 | \n", "Sovereign country | \n", "Albania | \n", "ALB | \n", "0 | \n", "... | \n", "Europe | \n", "Southern Europe | \n", "Europe & Central Asia | \n", "7 | \n", "7 | \n", "4 | \n", "-99 | \n", "1 | \n", "Admin-0 country | \n", "POLYGON ((20.59025 41.85540, 20.46318 41.51509... | \n", "
3 rows × 64 columns
\n", "