Skip to content

Commit 75b1fd6

Browse files
ishilandishiland
committed
add dockerfile and simple pandas example
1 parent ecd94e3 commit 75b1fd6

5 files changed

Lines changed: 149 additions & 9 deletions

File tree

‎examples/Dockerfile‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
FROM nycplanning/docker-geosupport:latest
2+
3+
RUN pip install pandas nyc-parser
4+
5+
WORKDIR /examples
6+
7+
ENTRYPOINT ["python"]

‎examples/data/input.csv‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
Borough,Address,Building ,Community Board,Council District ,Census Tract,BIN,BBL,NTA
2+
BRONX,1118 Grand Concourse,Bronx Housing Court,204,16,18102,2101266,2024620039,BX14
3+
BRONX,1118,Bronx Housing Court,204,16,18102,2101266,2024620039,BX14
4+
MANHATTAN,City Hall Park,City Hall,101,1,31,1000000,1001220001,MN24
5+
STATEN IS,100 Richmond Terrace,Staten Island Family Court,501,49,7,5000090,5000090022,SI22
6+
BROOKLYN,345 Adams St.,345 Adams St.,302,33,11,3392969,3001407503,BK38
7+
MANHATTAN,60 Centre St.,Manhattan Supreme Court,101,1,29,1085748,1001600021,MN27
8+
STATEN IS,18 Richmond Terrace,Staten Island Supreme Court,501,49,3,5000064,5000070012,SI22
9+
STATEN IS,927 Castleton Ave.,Staten Island Civil Court,501,49,105,5004530,5001590065,SI35
10+
BRONX,215 E. 161 St.,Bronx Family & Criminal Court,204,16,18302,2002704,2024540001,BX14
11+
MANHATTAN,253 Broadway,Home Life Building,101,1,21,1082757,1001347501,MN24
12+
BROOKLYN,390 Kent Ave.,DFMC Shops,301,33,549,3063635,3024670001,BK73
13+
MANHATTAN,115 Chrystie Street,Administration Building,103,1,18,1005645,1004230022,MN27
14+
MANHATTAN,22 Reade St.,City Planning Building,101,1,31,1078613,1001540023,MN24
15+
BRONX,851 Grand Concourse,Bronx County Courthouse,204,8,63,2002869,2024680001,BX63
16+
MANHATTAN,170 E. 121 St.,Harlem Courthouse,111,8,196,1054382,1017690045,MN34
17+
QUEENS,66-26 Metropolitan Ave.,Central Storehouse,405,30,61301,4443103,4036050001,QN20
18+
BROOKLYN,541 St Johns Place (COTA),OCA Academy,308,35,217,3389809,3011750001,BK61
19+
QUEENS,88-11 Sutphin Blvd.,Queens Supreme Court,412,24,238,4207071,4096910001,QN61
20+
BROOKLYN,120 Schermerhorn St.,Brooklyn Criminal Court,302,33,43,3000534,3001690017,BK38
21+
MANHATTAN,100 Centre St.,Manhattan Criminal Court,101,1,29,1079000,1001670001,MN27
22+
STATEN IS,10 Richmond Terrace,Staten Island Borough Hall,501,49,3,5000063,5000070001,SI22
23+
BRONX,265 E. 161 Street,Bronx Hall of Justice,204,16,18302,2097114,2024440001,BX14
24+
MANHATTAN,31 Chambers St.,Manhattan Surrogate's Court,101,1,31,1001670,1001530024,MN24
25+
BROOKLYN,210 Joralemon St.,Brooklyn Municipal Building,302,33,9,3002558,3002667501,BK09
26+
STATEN IS,130 Stuyvesant Pl.,130 Stuyvesant Pl.,501,49,3,5000085,5000080070,SI22
27+
QUEENS,120-55 Queens Blvd.,Queens Borough Hall,409,29,216,4052812,4022740002,QN60
28+
MANHATTAN,2 Lafayette St.,Court Square Building,101,1,31,1001672,1001550001,MN24
29+
MANHATTAN,52 Chambers St.,Tweed Courthouse,101,1,31,1079146,1001220001,MN24
30+
BRONX,1932 Arthur Ave.,Bergen Building,206,17,36901,2009911,2029470018,BX17
31+
QUEENS,125-01 Queens Blvd.,Queens Criminal Court,409,29,216,4206522,4096530001,QN60
32+
MANHATTAN,111 Centre St.,Manhattan Civil Court,101,1,31,1001833,1001690010,MN24
33+
BRONX,4101 White Plains Rd.,Bronx Neighborhood Building,212,12,420,2063174,2048320009,BX44
34+
BROOKLYN,330 Jay Street,Brooklyn Supreme & Family Court,302,33,11,3347736,3001407502,BK38
35+
MANHATTAN,280 Broadway,Sun Building,101,1,31,1079215,1001530001,MN24
36+
STATEN IS,26 Central Avenue,Staten Island Court,501,49,3,5151735,5000060021,SI22
37+
STATEN IS,67 Targee St. (vacant),Staten Island Criminal Court,501,49,21,5014078,5005420009,SI37
38+
QUEENS,151-20 Jamaica Ave.,New Queens Family Court,412,27,240,4438731,4100930001,QN61
39+
BRONX,198 E. 161 St.,Concourse Plaza,204,16,61,2099027,2024430094,BX14
40+
MANHATTAN,80 Centre St.,Louis Lefkowitz Building,101,1,29,1001830,1001660027,MN27
41+
BROOKLYN,209 Joralemon St.,Brooklyn Borough Hall,302,33,11,3000256,3001390001,BK38
42+
STATEN IS,111 Canal St. (vacant),Staten Island Village Hall,501,49,21,5013729,5005230001,SI37
43+
MANHATTAN,100 Gold St.,100 Gold St.,101,1,1501,1001289,1000940025,MN25
44+
QUEENS,25-10 Court House Square,Long Island City Courthouse,402,26,19,4000698,,QN31
45+
MANHATTAN,27 Madison Ave.,Manhattan Appellate Court,105,2,56,1016743,1008550001,MN13
46+
QUEENS,89-17 Sutphin Blvd.,Queens Civil Court,412,24,240,4448759,4096800001,QN61
47+
BROOKLYN,165 Cadman Plaza East,OEM Building,302,33,13,3000172,3000850006,BK38
48+
BROOKLYN,Brooklyn Navy Yard,OTS Building,302,33,543,3000000,3020230001,BK99
49+
MANHATTAN,314 W. 54th St.,Midtown Community Court,104,3,133,1025397,1010440022,MN15
50+
BROOKLYN,45 Monroe Pl.,Brooklyn Appellate Court,302,33,502,3001881,3002370001,BK09
51+
MANHATTAN,60 Lafayette St.,Manhattan Family Court,101,1,31,1001842,1001710031,MN24
52+
BROOKLYN,360 Adams St.,Brooklyn Supreme Court,302,33,11,3000257,3001390020,BK38
53+
MANHATTAN,1 Centre St.,Manhattan Municipal Building,101,1,29,1001394,1001210001,MN27
54+
BRONX,350 Marconi,PSACII (May 2016),211,13,284,2125035,2042260075,BX37
55+
MANHATTAN,125 Worth St.,Health Building,101,1,31,1001831,1001680032,MN24
56+
BROOKLYN,141 Livingston St. (Leased),Brooklyn Civil Court,302,33,37,3000428,3001540028,BK38
57+
QUEENS,126-02 82nd Avenue,Queens Family Justice Center,409,29,216,4458616,4096530001,QN60

examples/pandas-multiprocessing-example.py renamed to examples/pandas_multiprocessing.py

Lines changed: 14 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@
33
# Minor edits by torreyma: https://github.com/torreyma
44
#
55
from geosupport import Geosupport, GeosupportError
6+
from nycparser import Parser
67
import pandas as pd
78
from multiprocessing import Pool, cpu_count
89
from functools import partial
@@ -12,12 +13,14 @@
1213
Example of how to use python-geosupport, Pandas and Multiprocessing to speed up geocoding workflows.
1314
"""
1415

15-
# For Windows:
16-
g = Geosupport(geosupport_path="C:\\Program Files (x86)\\Geosupport Desktop Edition")
17-
# On linux, geosupport location is set in environment variables GEOFILES and LD_LIBRARY_PATH.
16+
g = Geosupport()
17+
p = Parser()
1818

1919
cpus = cpu_count()
2020

21+
INPUT_CSV = '/examples/data/input.csv'
22+
OUTPUT_CSV = '/examples/data/output-pandas-multiprocessing.csv'
23+
2124

2225
def geo_by_address(row):
2326
"""
@@ -27,15 +30,17 @@ def geo_by_address(row):
2730
:return: Pandas Series with lat, lon & Geosupport message.
2831
"""
2932
try:
30-
result = g.address(house_number=row['PHN'], street_name=row['STREET'], zip=row['ZIP_CODE']) # Adjust these to match your data column names
33+
# parse the address to separate PHN and street
34+
parsed = p.address(row['Address'])
35+
# geocode
36+
result = g.address(house_number=parsed['PHN'], street_name=parsed['STREET'], borough=row['Borough'])
3137
lat = result.get("Latitude")
3238
lon = result.get('Longitude')
3339
msg = result.get('Message')
3440
except GeosupportError as ge:
35-
lat = "Error"
36-
lon = "Error"
41+
lat = ""
42+
lon = ""
3743
msg = str(ge)
38-
pass
3944
return pd.Series([lat, lon, msg])
4045

4146

@@ -59,13 +64,13 @@ def parallelize_on_rows(data, func, num_of_processes=cpus):
5964
if __name__ == '__main__':
6065

6166
# read in csv
62-
df = pd.read_csv('INPUT.csv')
67+
df = pd.read_csv(INPUT_CSV)
6368

6469
# add 3 Geosupport columns - Latitude, Longitude and Geosupport message
6570
df[['lat', 'lon', 'msg']] = parallelize_on_rows(df, geo_by_address)
6671

6772
# output to csv with the 3 new columns.
68-
df.to_csv('OUTPUT.csv')
73+
df.to_csv(OUTPUT_CSV)
6974

7075

7176

‎examples/pandas_simple.py‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
from geosupport import Geosupport, GeosupportError
2+
from nycparser import Parser
3+
import pandas as pd
4+
from multiprocessing import Pool, cpu_count
5+
from functools import partial
6+
import numpy as np
7+
8+
"""
9+
Example of how to use python-geosupport and pandas dataframes.
10+
"""
11+
12+
g = Geosupport()
13+
p = Parser()
14+
15+
INPUT_CSV = '/examples/data/input.csv'
16+
OUTPUT_CSV = '/examples/data/output-pandas-simple.csv'
17+
18+
19+
def geo_by_address(row):
20+
"""
21+
Geocodes a pandas row containing address attributes.
22+
:param row: Pandas Series
23+
:return: Pandas Series with lat, lng & Geosupport message.
24+
"""
25+
try:
26+
# parse the address to separate PHN and street
27+
parsed = p.address(row['Address'])
28+
# geocode
29+
result = g.address(house_number=parsed['PHN'], street_name=parsed['STREET'], borough=row['Borough'])
30+
lat = result.get("Latitude")
31+
lon = result.get('Longitude')
32+
msg = result.get('Message')
33+
except GeosupportError as ge:
34+
lat = ""
35+
lon = ""
36+
msg = str(ge)
37+
return pd.Series([lat, lon, msg])
38+
39+
40+
if __name__ == '__main__':
41+
# read in csv
42+
df = pd.read_csv(INPUT_CSV)
43+
44+
# add 3 Geosupport columns - Latitude, Longitude and Geosupport message
45+
df[['lat', 'lon', 'msg']] = df.apply(geo_by_address, axis=1)
46+
47+
# output the new dataframe to a csv
48+
df.to_csv(OUTPUT_CSV, index=False)

‎examples/readme.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Geosupport Examples
2+
3+
Some examples to demonstrate different usages of python-geosupport.
4+
5+
## Using Docker
6+
Use NYC Plannings [docker-geosupport](https://github.com/NYCPlanning/docker-geosupport) image to geocode your data.
7+
8+
> nycplanning/docker-geosupport image will be automatically updated whenever there's a new major release or upad release on Bytes of the Big Apple
9+
10+
### Build
11+
```shell
12+
docker build . -t geosupport-examples
13+
```
14+
15+
### Run
16+
From this directory, you can run:
17+
```shell
18+
docker run -it --rm --volume ${PWD}:/examples geosupport-examples pandas_simple.py
19+
20+
docker run -it --rm --volume ${PWD}:/examples geosupport-examples pandas_multiprocessing.py
21+
```
22+
23+
The outputs will be in the *./data* directory.

0 commit comments

Comments
 (0)