edegan.com - User contributions [en]

Using R in PostgreSQL

2018-03-20T17:26:26Z

JeeminS: /* Sample call of an aggregate function */

== Perhaps a Simpler Alternative ==
Aggregate functions in PostgreSQL:
https://www.postgresql.org/docs/9.0/static/functions-aggregate.html

=== Aggregate Functions ===

Table 9-44. Aggregate Functions for Statistics

Function | Argument Type | Return Type | Description
=======================+==========================================================+=================================================================+=============================================================================================
corr(Y, X) | double precision | double precision | correlation coefficient
covar_pop(Y, X) | double precision | double precision | population covariance
covar_samp(Y, X) | double precision | double precision | sample covariance
regr_avgx(Y, X) | double precision | double precision | average of the independent variable (sum(X)/N)
regr_avgy(Y, X) | double precision | double precision | average of the dependent variable (sum(Y)/N)
regr_count(Y, X) | double precision | bigint | number of input rows in which both expressions are nonnull
regr_intercept(Y, X) | double precision | double precision | y-intercept of the least-squares-fit linear equation determined by the (X, Y) pairs
regr_r2(Y, X) | double precision | double precision | square of the correlation coefficient
regr_slope(Y, X) | double precision | double precision | slope of the least-squares-fit linear equation determined by the (X, Y) pairs
regr_sxx(Y, X) | double precision | double precision | sum(X^2) - sum(X)^2/N ("sum of squares" of the independent variable)
regr_sxy(Y, X) | double precision | double precision | sum(X*Y) - sum(X) * sum(Y)/N ("sum of products" of independent times dependent variable)
regr_syy(Y, X) | double precision | double precision |sum(Y^2) - sum(Y)^2/N ("sum of squares" of the dependent variable)
stddev(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| historical alias for stddev_samp
stddev_pop(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| population standard deviation of the input values
stddev_samp(expression)| smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| sample standard deviation of the input values
variance(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| historical alias for var_samp
var_pop(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| population variance of the input values (square of the population standard deviation)
var_samp(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| sample variance of the input values (square of the sample standard deviation)

=== Sample call of an aggregate function ===

SELECT regr_slope(tothullcount, avghullcount) FROM hcllayerwinv;

==== Output ====
regr_slope
------------------
2.31850202053387
(1 row)

==Joe Conway's Documentation on pl/r==

http://joeconway.com/doc/plr-US.pdf
http://www.joeconway.com/plr/doc/plr-funcs.html
http://www.joeconway.com/plr/doc/plr-window-funcs.html

=== Sample Function Code in PostgreSQL ===

==== Creating the Sample Function ====
Inside a database (ex) tigertest):

CREATE OR REPLACE FUNCTION r_max (integer, integer) RETURNS integer AS ’
if (arg1 > arg2)
return(arg1)
else
return(arg2)
’ LANGUAGE ’plr’ STRICT;

==== Calling the Sample Function ====

SELECT * FROM r_max(2,3);

returns

r_max
-------
3
(1 row)

=== Attempts to call R built-in function from PSQL ===

==== Regression ====

Creating Function:

CREATE OR REPLACE FUNCTION r_reg(y numeric[], x numeric[])
RETURNS numeric AS $$
res <- lm(y ~ x)
return(coef(res)[1])
$$ LANGUAGE 'plr' immutable;

Calling Function:

SELECT r_reg(tothullcount, avghullcount) from hcllayerwinv;

Results:

r_reg
-------
0
0
3
0
5
4
0
0
5
4
0
0
6
4
4
0
0
4
0
0
4
0
0
4
0
0
3
0
5
3
0
0
7
5
3
3
0
0
6
4
0
0
0
5
4
0
0
7

== hcllayerwinv table from tigertest database ==

Table "public.hcllayerwinv"
Column | Type | Collation | Nullable | Default
------------------+------------------------+-----------+----------+---------
place | character varying(100) | | |
statecode | character varying(2) | | |
year | integer | | |
layer | integer | | |
lcount | numeric | | |
nosingleton | bigint | | |
nomultiton | bigint | | |
nopair | bigint | | |
nohull | bigint | | |
totmultitoncount | numeric | | |
avgmultitoncount | numeric | | |
totpaircount | numeric | | |
avgpaircount | numeric | | |
totpairlength | double precision | | |
avgpairlength | double precision | | |
minpairlength | double precision | | |
maxpairlength | double precision | | |
tothullcount | numeric | | |
avghullcount | numeric | | |
tothullarea | double precision | | |
avghullarea | double precision | | |
minhullarea | double precision | | |
maxhullarea | double precision | | |
tothulldensity | double precision | | |
avghulldensity | double precision | | |
seedearlyinvf | double precision | | |
inflator | numeric | | |
seedearlyinvl16f | double precision | | |

=== Regress by===
--By: place, statecode, layer,
--Regress: seedearlyinvl16f ON nosingleton, totmultitoncount, totpaircount, tothullcount, avgpairlength, avghullarea, avghulldensity

==Installing PL/R on the dbase server==

Check the version of the dbase by doing a
SELECT version();

This gives us 9.5.10

Then as root:
apt-get install postgresql-9.5-plr

Finally enable the extension in a dbase:
create extension plr;

See also: http://www.joeconway.com/presentations/plr-PGConfNYC2014.pdf

==Instructions on Use==

To use R from pgAdmin III, follow the instructions in [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial]: choose the database, click SQL, and run "CREATE EXTENSION plr;". This allows R functions to be used in this database from then on. You should be able to run the given examples in the tutorial after this is done. This was run on databases tigertest and template1.

[https://www.joeconway.com/presentations/plr-DWDC-2015.05.pdf Another possibly useful presentation on PL/R.]

==How PL/R was installed on the RDP==

On 2018-01-18, PL/R was installed to allow PostgreSQL users to use R functions from SQL queries. It was installed using [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial.] The link from the tutorial for PL/R doesn't work, I used [http://www.joeconway.com/plr.html this instead.]

It required R version 3.3.0 for PL/R to work with PostgreSQL 9.5. If the environmental variables R_HOME or PATH are edited so that they point to a different version of R, this might not work, and PL/R might have to be reinstalled. If the version of PostgreSQL is updated, both the R version and PL/R version will have to match it as well.

Using R in PostgreSQL

2018-03-20T17:24:44Z

JeeminS: /* Perhaps a Simpler Alternative */

== Perhaps a Simpler Alternative ==
Aggregate functions in PostgreSQL:
https://www.postgresql.org/docs/9.0/static/functions-aggregate.html

=== Aggregate Functions ===

Table 9-44. Aggregate Functions for Statistics

Function | Argument Type | Return Type | Description
=======================+==========================================================+=================================================================+=============================================================================================
corr(Y, X) | double precision | double precision | correlation coefficient
covar_pop(Y, X) | double precision | double precision | population covariance
covar_samp(Y, X) | double precision | double precision | sample covariance
regr_avgx(Y, X) | double precision | double precision | average of the independent variable (sum(X)/N)
regr_avgy(Y, X) | double precision | double precision | average of the dependent variable (sum(Y)/N)
regr_count(Y, X) | double precision | bigint | number of input rows in which both expressions are nonnull
regr_intercept(Y, X) | double precision | double precision | y-intercept of the least-squares-fit linear equation determined by the (X, Y) pairs
regr_r2(Y, X) | double precision | double precision | square of the correlation coefficient
regr_slope(Y, X) | double precision | double precision | slope of the least-squares-fit linear equation determined by the (X, Y) pairs
regr_sxx(Y, X) | double precision | double precision | sum(X^2) - sum(X)^2/N ("sum of squares" of the independent variable)
regr_sxy(Y, X) | double precision | double precision | sum(X*Y) - sum(X) * sum(Y)/N ("sum of products" of independent times dependent variable)
regr_syy(Y, X) | double precision | double precision |sum(Y^2) - sum(Y)^2/N ("sum of squares" of the dependent variable)
stddev(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| historical alias for stddev_samp
stddev_pop(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| population standard deviation of the input values
stddev_samp(expression)| smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| sample standard deviation of the input values
variance(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| historical alias for var_samp
var_pop(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| population variance of the input values (square of the population standard deviation)
var_samp(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| sample variance of the input values (square of the sample standard deviation)

=== Sample call of an aggregate function ===

SELECT regr_slope(tothullcount, avghullcount) FROM hcllayerwinv;
regr_slope
------------------
2.31850202053387
(1 row)

==Joe Conway's Documentation on pl/r==

http://joeconway.com/doc/plr-US.pdf
http://www.joeconway.com/plr/doc/plr-funcs.html
http://www.joeconway.com/plr/doc/plr-window-funcs.html

=== Sample Function Code in PostgreSQL ===

==== Creating the Sample Function ====
Inside a database (ex) tigertest):

CREATE OR REPLACE FUNCTION r_max (integer, integer) RETURNS integer AS ’
if (arg1 > arg2)
return(arg1)
else
return(arg2)
’ LANGUAGE ’plr’ STRICT;

==== Calling the Sample Function ====

SELECT * FROM r_max(2,3);

returns

r_max
-------
3
(1 row)

=== Attempts to call R built-in function from PSQL ===

==== Regression ====

Creating Function:

CREATE OR REPLACE FUNCTION r_reg(y numeric[], x numeric[])
RETURNS numeric AS $$
res <- lm(y ~ x)
return(coef(res)[1])
$$ LANGUAGE 'plr' immutable;

Calling Function:

SELECT r_reg(tothullcount, avghullcount) from hcllayerwinv;

Results:

r_reg
-------
0
0
3
0
5
4
0
0
5
4
0
0
6
4
4
0
0
4
0
0
4
0
0
4
0
0
3
0
5
3
0
0
7
5
3
3
0
0
6
4
0
0
0
5
4
0
0
7

== hcllayerwinv table from tigertest database ==

Table "public.hcllayerwinv"
Column | Type | Collation | Nullable | Default
------------------+------------------------+-----------+----------+---------
place | character varying(100) | | |
statecode | character varying(2) | | |
year | integer | | |
layer | integer | | |
lcount | numeric | | |
nosingleton | bigint | | |
nomultiton | bigint | | |
nopair | bigint | | |
nohull | bigint | | |
totmultitoncount | numeric | | |
avgmultitoncount | numeric | | |
totpaircount | numeric | | |
avgpaircount | numeric | | |
totpairlength | double precision | | |
avgpairlength | double precision | | |
minpairlength | double precision | | |
maxpairlength | double precision | | |
tothullcount | numeric | | |
avghullcount | numeric | | |
tothullarea | double precision | | |
avghullarea | double precision | | |
minhullarea | double precision | | |
maxhullarea | double precision | | |
tothulldensity | double precision | | |
avghulldensity | double precision | | |
seedearlyinvf | double precision | | |
inflator | numeric | | |
seedearlyinvl16f | double precision | | |

=== Regress by===
--By: place, statecode, layer,
--Regress: seedearlyinvl16f ON nosingleton, totmultitoncount, totpaircount, tothullcount, avgpairlength, avghullarea, avghulldensity

==Installing PL/R on the dbase server==

Check the version of the dbase by doing a
SELECT version();

This gives us 9.5.10

Then as root:
apt-get install postgresql-9.5-plr

Finally enable the extension in a dbase:
create extension plr;

See also: http://www.joeconway.com/presentations/plr-PGConfNYC2014.pdf

==Instructions on Use==

To use R from pgAdmin III, follow the instructions in [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial]: choose the database, click SQL, and run "CREATE EXTENSION plr;". This allows R functions to be used in this database from then on. You should be able to run the given examples in the tutorial after this is done. This was run on databases tigertest and template1.

[https://www.joeconway.com/presentations/plr-DWDC-2015.05.pdf Another possibly useful presentation on PL/R.]

==How PL/R was installed on the RDP==

On 2018-01-18, PL/R was installed to allow PostgreSQL users to use R functions from SQL queries. It was installed using [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial.] The link from the tutorial for PL/R doesn't work, I used [http://www.joeconway.com/plr.html this instead.]

It required R version 3.3.0 for PL/R to work with PostgreSQL 9.5. If the environmental variables R_HOME or PATH are edited so that they point to a different version of R, this might not work, and PL/R might have to be reinstalled. If the version of PostgreSQL is updated, both the R version and PL/R version will have to match it as well.

Using R in PostgreSQL

2018-03-20T17:22:28Z

JeeminS: /* Regression */

== Perhaps a Simpler Alternative ==
Aggregate functions in PostgreSQL:
https://www.postgresql.org/docs/9.0/static/functions-aggregate.html

=== Aggregate Functions ===

Table 9-44. Aggregate Functions for Statistics

Function | Argument Type | Return Type | Description
=======================+==========================================================+=================================================================+=============================================================================================
corr(Y, X) | double precision | double precision | correlation coefficient
covar_pop(Y, X) | double precision | double precision | population covariance
covar_samp(Y, X) | double precision | double precision | sample covariance
regr_avgx(Y, X) | double precision | double precision | average of the independent variable (sum(X)/N)
regr_avgy(Y, X) | double precision | double precision | average of the dependent variable (sum(Y)/N)
regr_count(Y, X) | double precision | bigint | number of input rows in which both expressions are nonnull
regr_intercept(Y, X) | double precision | double precision | y-intercept of the least-squares-fit linear equation determined by the (X, Y) pairs
regr_r2(Y, X) | double precision | double precision | square of the correlation coefficient
regr_slope(Y, X) | double precision | double precision | slope of the least-squares-fit linear equation determined by the (X, Y) pairs
regr_sxx(Y, X) | double precision | double precision | sum(X^2) - sum(X)^2/N ("sum of squares" of the independent variable)
regr_sxy(Y, X) | double precision | double precision | sum(X*Y) - sum(X) * sum(Y)/N ("sum of products" of independent times dependent variable)
regr_syy(Y, X) | double precision | double precision |sum(Y^2) - sum(Y)^2/N ("sum of squares" of the dependent variable)
stddev(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| historical alias for stddev_samp
stddev_pop(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| population standard deviation of the input values
stddev_samp(expression)| smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| sample standard deviation of the input values
variance(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| historical alias for var_samp
var_pop(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| population variance of the input values (square of the population standard deviation)
var_samp(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| sample variance of the input values (square of the sample standard deviation)

==Joe Conway's Documentation on pl/r==

http://joeconway.com/doc/plr-US.pdf
http://www.joeconway.com/plr/doc/plr-funcs.html
http://www.joeconway.com/plr/doc/plr-window-funcs.html

=== Sample Function Code in PostgreSQL ===

==== Creating the Sample Function ====
Inside a database (ex) tigertest):

CREATE OR REPLACE FUNCTION r_max (integer, integer) RETURNS integer AS ’
if (arg1 > arg2)
return(arg1)
else
return(arg2)
’ LANGUAGE ’plr’ STRICT;

==== Calling the Sample Function ====

SELECT * FROM r_max(2,3);

returns

r_max
-------
3
(1 row)

=== Attempts to call R built-in function from PSQL ===

==== Regression ====

Creating Function:

CREATE OR REPLACE FUNCTION r_reg(y numeric[], x numeric[])
RETURNS numeric AS $$
res <- lm(y ~ x)
return(coef(res)[1])
$$ LANGUAGE 'plr' immutable;

Calling Function:

SELECT r_reg(tothullcount, avghullcount) from hcllayerwinv;

Results:

r_reg
-------
0
0
3
0
5
4
0
0
5
4
0
0
6
4
4
0
0
4
0
0
4
0
0
4
0
0
3
0
5
3
0
0
7
5
3
3
0
0
6
4
0
0
0
5
4
0
0
7

== hcllayerwinv table from tigertest database ==

Table "public.hcllayerwinv"
Column | Type | Collation | Nullable | Default
------------------+------------------------+-----------+----------+---------
place | character varying(100) | | |
statecode | character varying(2) | | |
year | integer | | |
layer | integer | | |
lcount | numeric | | |
nosingleton | bigint | | |
nomultiton | bigint | | |
nopair | bigint | | |
nohull | bigint | | |
totmultitoncount | numeric | | |
avgmultitoncount | numeric | | |
totpaircount | numeric | | |
avgpaircount | numeric | | |
totpairlength | double precision | | |
avgpairlength | double precision | | |
minpairlength | double precision | | |
maxpairlength | double precision | | |
tothullcount | numeric | | |
avghullcount | numeric | | |
tothullarea | double precision | | |
avghullarea | double precision | | |
minhullarea | double precision | | |
maxhullarea | double precision | | |
tothulldensity | double precision | | |
avghulldensity | double precision | | |
seedearlyinvf | double precision | | |
inflator | numeric | | |
seedearlyinvl16f | double precision | | |

=== Regress by===
--By: place, statecode, layer,
--Regress: seedearlyinvl16f ON nosingleton, totmultitoncount, totpaircount, tothullcount, avgpairlength, avghullarea, avghulldensity

==Installing PL/R on the dbase server==

Check the version of the dbase by doing a
SELECT version();

This gives us 9.5.10

Then as root:
apt-get install postgresql-9.5-plr

Finally enable the extension in a dbase:
create extension plr;

See also: http://www.joeconway.com/presentations/plr-PGConfNYC2014.pdf

==Instructions on Use==

To use R from pgAdmin III, follow the instructions in [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial]: choose the database, click SQL, and run "CREATE EXTENSION plr;". This allows R functions to be used in this database from then on. You should be able to run the given examples in the tutorial after this is done. This was run on databases tigertest and template1.

[https://www.joeconway.com/presentations/plr-DWDC-2015.05.pdf Another possibly useful presentation on PL/R.]

==How PL/R was installed on the RDP==

On 2018-01-18, PL/R was installed to allow PostgreSQL users to use R functions from SQL queries. It was installed using [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial.] The link from the tutorial for PL/R doesn't work, I used [http://www.joeconway.com/plr.html this instead.]

It required R version 3.3.0 for PL/R to work with PostgreSQL 9.5. If the environmental variables R_HOME or PATH are edited so that they point to a different version of R, this might not work, and PL/R might have to be reinstalled. If the version of PostgreSQL is updated, both the R version and PL/R version will have to match it as well.

Using R in PostgreSQL

2018-03-20T17:18:09Z

JeeminS: /* Attempts to call R built-in function from PSQL */

== Perhaps a Simpler Alternative ==
Aggregate functions in PostgreSQL:
https://www.postgresql.org/docs/9.0/static/functions-aggregate.html

=== Aggregate Functions ===

Table 9-44. Aggregate Functions for Statistics

Function | Argument Type | Return Type | Description
=======================+==========================================================+=================================================================+=============================================================================================
corr(Y, X) | double precision | double precision | correlation coefficient
covar_pop(Y, X) | double precision | double precision | population covariance
covar_samp(Y, X) | double precision | double precision | sample covariance
regr_avgx(Y, X) | double precision | double precision | average of the independent variable (sum(X)/N)
regr_avgy(Y, X) | double precision | double precision | average of the dependent variable (sum(Y)/N)
regr_count(Y, X) | double precision | bigint | number of input rows in which both expressions are nonnull
regr_intercept(Y, X) | double precision | double precision | y-intercept of the least-squares-fit linear equation determined by the (X, Y) pairs
regr_r2(Y, X) | double precision | double precision | square of the correlation coefficient
regr_slope(Y, X) | double precision | double precision | slope of the least-squares-fit linear equation determined by the (X, Y) pairs
regr_sxx(Y, X) | double precision | double precision | sum(X^2) - sum(X)^2/N ("sum of squares" of the independent variable)
regr_sxy(Y, X) | double precision | double precision | sum(X*Y) - sum(X) * sum(Y)/N ("sum of products" of independent times dependent variable)
regr_syy(Y, X) | double precision | double precision |sum(Y^2) - sum(Y)^2/N ("sum of squares" of the dependent variable)
stddev(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| historical alias for stddev_samp
stddev_pop(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| population standard deviation of the input values
stddev_samp(expression)| smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| sample standard deviation of the input values
variance(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| historical alias for var_samp
var_pop(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| population variance of the input values (square of the population standard deviation)
var_samp(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| sample variance of the input values (square of the sample standard deviation)

==Joe Conway's Documentation on pl/r==

http://joeconway.com/doc/plr-US.pdf
http://www.joeconway.com/plr/doc/plr-funcs.html
http://www.joeconway.com/plr/doc/plr-window-funcs.html

=== Sample Function Code in PostgreSQL ===

==== Creating the Sample Function ====
Inside a database (ex) tigertest):

CREATE OR REPLACE FUNCTION r_max (integer, integer) RETURNS integer AS ’
if (arg1 > arg2)
return(arg1)
else
return(arg2)
’ LANGUAGE ’plr’ STRICT;

==== Calling the Sample Function ====

SELECT * FROM r_max(2,3);

returns

r_max
-------
3
(1 row)

=== Attempts to call R built-in function from PSQL ===

==== Regression ====

Creating Function:

CREATE OR REPLACE FUNCTION r_reg(y numeric[], x numeric[])
RETURNS numeric AS $$
res <- lm(y ~ x)
return(coef(res)[1])
$$ LANGUAGE 'plr' immutable;

Calling Function:

SELECT *

== hcllayerwinv table from tigertest database ==

Table "public.hcllayerwinv"
Column | Type | Collation | Nullable | Default
------------------+------------------------+-----------+----------+---------
place | character varying(100) | | |
statecode | character varying(2) | | |
year | integer | | |
layer | integer | | |
lcount | numeric | | |
nosingleton | bigint | | |
nomultiton | bigint | | |
nopair | bigint | | |
nohull | bigint | | |
totmultitoncount | numeric | | |
avgmultitoncount | numeric | | |
totpaircount | numeric | | |
avgpaircount | numeric | | |
totpairlength | double precision | | |
avgpairlength | double precision | | |
minpairlength | double precision | | |
maxpairlength | double precision | | |
tothullcount | numeric | | |
avghullcount | numeric | | |
tothullarea | double precision | | |
avghullarea | double precision | | |
minhullarea | double precision | | |
maxhullarea | double precision | | |
tothulldensity | double precision | | |
avghulldensity | double precision | | |
seedearlyinvf | double precision | | |
inflator | numeric | | |
seedearlyinvl16f | double precision | | |

=== Regress by===
--By: place, statecode, layer,
--Regress: seedearlyinvl16f ON nosingleton, totmultitoncount, totpaircount, tothullcount, avgpairlength, avghullarea, avghulldensity

==Installing PL/R on the dbase server==

Check the version of the dbase by doing a
SELECT version();

This gives us 9.5.10

Then as root:
apt-get install postgresql-9.5-plr

Finally enable the extension in a dbase:
create extension plr;

See also: http://www.joeconway.com/presentations/plr-PGConfNYC2014.pdf

==Instructions on Use==

To use R from pgAdmin III, follow the instructions in [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial]: choose the database, click SQL, and run "CREATE EXTENSION plr;". This allows R functions to be used in this database from then on. You should be able to run the given examples in the tutorial after this is done. This was run on databases tigertest and template1.

[https://www.joeconway.com/presentations/plr-DWDC-2015.05.pdf Another possibly useful presentation on PL/R.]

==How PL/R was installed on the RDP==

On 2018-01-18, PL/R was installed to allow PostgreSQL users to use R functions from SQL queries. It was installed using [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial.] The link from the tutorial for PL/R doesn't work, I used [http://www.joeconway.com/plr.html this instead.]

It required R version 3.3.0 for PL/R to work with PostgreSQL 9.5. If the environmental variables R_HOME or PATH are edited so that they point to a different version of R, this might not work, and PL/R might have to be reinstalled. If the version of PostgreSQL is updated, both the R version and PL/R version will have to match it as well.

Using R in PostgreSQL

2018-03-20T17:14:29Z

JeeminS: /* Joe Conway's Documentation on pl/r */

== Perhaps a Simpler Alternative ==
Aggregate functions in PostgreSQL:
https://www.postgresql.org/docs/9.0/static/functions-aggregate.html

=== Aggregate Functions ===

Table 9-44. Aggregate Functions for Statistics

Function | Argument Type | Return Type | Description
=======================+==========================================================+=================================================================+=============================================================================================
corr(Y, X) | double precision | double precision | correlation coefficient
covar_pop(Y, X) | double precision | double precision | population covariance
covar_samp(Y, X) | double precision | double precision | sample covariance
regr_avgx(Y, X) | double precision | double precision | average of the independent variable (sum(X)/N)
regr_avgy(Y, X) | double precision | double precision | average of the dependent variable (sum(Y)/N)
regr_count(Y, X) | double precision | bigint | number of input rows in which both expressions are nonnull
regr_intercept(Y, X) | double precision | double precision | y-intercept of the least-squares-fit linear equation determined by the (X, Y) pairs
regr_r2(Y, X) | double precision | double precision | square of the correlation coefficient
regr_slope(Y, X) | double precision | double precision | slope of the least-squares-fit linear equation determined by the (X, Y) pairs
regr_sxx(Y, X) | double precision | double precision | sum(X^2) - sum(X)^2/N ("sum of squares" of the independent variable)
regr_sxy(Y, X) | double precision | double precision | sum(X*Y) - sum(X) * sum(Y)/N ("sum of products" of independent times dependent variable)
regr_syy(Y, X) | double precision | double precision |sum(Y^2) - sum(Y)^2/N ("sum of squares" of the dependent variable)
stddev(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| historical alias for stddev_samp
stddev_pop(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| population standard deviation of the input values
stddev_samp(expression)| smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| sample standard deviation of the input values
variance(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| historical alias for var_samp
var_pop(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| population variance of the input values (square of the population standard deviation)
var_samp(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| sample variance of the input values (square of the sample standard deviation)

==Joe Conway's Documentation on pl/r==

http://joeconway.com/doc/plr-US.pdf
http://www.joeconway.com/plr/doc/plr-funcs.html
http://www.joeconway.com/plr/doc/plr-window-funcs.html

=== Sample Function Code in PostgreSQL ===

==== Creating the Sample Function ====
Inside a database (ex) tigertest):

CREATE OR REPLACE FUNCTION r_max (integer, integer) RETURNS integer AS ’
if (arg1 > arg2)
return(arg1)
else
return(arg2)
’ LANGUAGE ’plr’ STRICT;

==== Calling the Sample Function ====

SELECT * FROM r_max(2,3);

returns

r_max
-------
3
(1 row)

=== Attempts to call R built-in function from PSQL ===

== hcllayerwinv table from tigertest database ==

Table "public.hcllayerwinv"
Column | Type | Collation | Nullable | Default
------------------+------------------------+-----------+----------+---------
place | character varying(100) | | |
statecode | character varying(2) | | |
year | integer | | |
layer | integer | | |
lcount | numeric | | |
nosingleton | bigint | | |
nomultiton | bigint | | |
nopair | bigint | | |
nohull | bigint | | |
totmultitoncount | numeric | | |
avgmultitoncount | numeric | | |
totpaircount | numeric | | |
avgpaircount | numeric | | |
totpairlength | double precision | | |
avgpairlength | double precision | | |
minpairlength | double precision | | |
maxpairlength | double precision | | |
tothullcount | numeric | | |
avghullcount | numeric | | |
tothullarea | double precision | | |
avghullarea | double precision | | |
minhullarea | double precision | | |
maxhullarea | double precision | | |
tothulldensity | double precision | | |
avghulldensity | double precision | | |
seedearlyinvf | double precision | | |
inflator | numeric | | |
seedearlyinvl16f | double precision | | |

=== Regress by===
--By: place, statecode, layer,
--Regress: seedearlyinvl16f ON nosingleton, totmultitoncount, totpaircount, tothullcount, avgpairlength, avghullarea, avghulldensity

==Installing PL/R on the dbase server==

Check the version of the dbase by doing a
SELECT version();

This gives us 9.5.10

Then as root:
apt-get install postgresql-9.5-plr

Finally enable the extension in a dbase:
create extension plr;

See also: http://www.joeconway.com/presentations/plr-PGConfNYC2014.pdf

==Instructions on Use==

To use R from pgAdmin III, follow the instructions in [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial]: choose the database, click SQL, and run "CREATE EXTENSION plr;". This allows R functions to be used in this database from then on. You should be able to run the given examples in the tutorial after this is done. This was run on databases tigertest and template1.

[https://www.joeconway.com/presentations/plr-DWDC-2015.05.pdf Another possibly useful presentation on PL/R.]

==How PL/R was installed on the RDP==

On 2018-01-18, PL/R was installed to allow PostgreSQL users to use R functions from SQL queries. It was installed using [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial.] The link from the tutorial for PL/R doesn't work, I used [http://www.joeconway.com/plr.html this instead.]

It required R version 3.3.0 for PL/R to work with PostgreSQL 9.5. If the environmental variables R_HOME or PATH are edited so that they point to a different version of R, this might not work, and PL/R might have to be reinstalled. If the version of PostgreSQL is updated, both the R version and PL/R version will have to match it as well.

Using R in PostgreSQL

2018-03-06T19:25:48Z

JeeminS: /* Aggregate Functions */

== Perhaps a Simpler Alternative ==
Aggregate functions in PostgreSQL:
https://www.postgresql.org/docs/9.0/static/functions-aggregate.html

=== Aggregate Functions ===

Table 9-44. Aggregate Functions for Statistics

Function | Argument Type | Return Type | Description
=======================+==========================================================+=================================================================+=============================================================================================
corr(Y, X) | double precision | double precision | correlation coefficient
covar_pop(Y, X) | double precision | double precision | population covariance
covar_samp(Y, X) | double precision | double precision | sample covariance
regr_avgx(Y, X) | double precision | double precision | average of the independent variable (sum(X)/N)
regr_avgy(Y, X) | double precision | double precision | average of the dependent variable (sum(Y)/N)
regr_count(Y, X) | double precision | bigint | number of input rows in which both expressions are nonnull
regr_intercept(Y, X) | double precision | double precision | y-intercept of the least-squares-fit linear equation determined by the (X, Y) pairs
regr_r2(Y, X) | double precision | double precision | square of the correlation coefficient
regr_slope(Y, X) | double precision | double precision | slope of the least-squares-fit linear equation determined by the (X, Y) pairs
regr_sxx(Y, X) | double precision | double precision | sum(X^2) - sum(X)^2/N ("sum of squares" of the independent variable)
regr_sxy(Y, X) | double precision | double precision | sum(X*Y) - sum(X) * sum(Y)/N ("sum of products" of independent times dependent variable)
regr_syy(Y, X) | double precision | double precision |sum(Y^2) - sum(Y)^2/N ("sum of squares" of the dependent variable)
stddev(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| historical alias for stddev_samp
stddev_pop(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| population standard deviation of the input values
stddev_samp(expression)| smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| sample standard deviation of the input values
variance(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| historical alias for var_samp
var_pop(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| population variance of the input values (square of the population standard deviation)
var_samp(expression) | smallint, int, bigint, real, double precision, or numeric| double precision for floating-point arguments, otherwise numeric| sample variance of the input values (square of the sample standard deviation)

==Joe Conway's Documentation on pl/r==

http://joeconway.com/doc/plr-US.pdf
http://www.joeconway.com/plr/doc/plr-funcs.html
http://www.joeconway.com/plr/doc/plr-window-funcs.html

=== Sample Function Code in PostgreSQL ===

==== Creating the Sample Function ====
Inside a database (ex) tigertest):

CREATE OR REPLACE FUNCTION r_max (integer, integer) RETURNS integer AS ’
if (arg1 > arg2)
return(arg1)
else
return(arg2)
’ LANGUAGE ’plr’ STRICT;

==== Calling the Sample Function ====

SELECT * FROM r_max(2,3);

returns

r_max
-------
3
(1 row)

== hcllayerwinv table from tigertest database ==

Table "public.hcllayerwinv"
Column | Type | Collation | Nullable | Default
------------------+------------------------+-----------+----------+---------
place | character varying(100) | | |
statecode | character varying(2) | | |
year | integer | | |
layer | integer | | |
lcount | numeric | | |
nosingleton | bigint | | |
nomultiton | bigint | | |
nopair | bigint | | |
nohull | bigint | | |
totmultitoncount | numeric | | |
avgmultitoncount | numeric | | |
totpaircount | numeric | | |
avgpaircount | numeric | | |
totpairlength | double precision | | |
avgpairlength | double precision | | |
minpairlength | double precision | | |
maxpairlength | double precision | | |
tothullcount | numeric | | |
avghullcount | numeric | | |
tothullarea | double precision | | |
avghullarea | double precision | | |
minhullarea | double precision | | |
maxhullarea | double precision | | |
tothulldensity | double precision | | |
avghulldensity | double precision | | |
seedearlyinvf | double precision | | |
inflator | numeric | | |
seedearlyinvl16f | double precision | | |

=== Regress by===
--By: place, statecode, layer,
--Regress: seedearlyinvl16f ON nosingleton, totmultitoncount, totpaircount, tothullcount, avgpairlength, avghullarea, avghulldensity

==Installing PL/R on the dbase server==

Check the version of the dbase by doing a
SELECT version();

This gives us 9.5.10

Then as root:
apt-get install postgresql-9.5-plr

Finally enable the extension in a dbase:
create extension plr;

See also: http://www.joeconway.com/presentations/plr-PGConfNYC2014.pdf

==Instructions on Use==

To use R from pgAdmin III, follow the instructions in [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial]: choose the database, click SQL, and run "CREATE EXTENSION plr;". This allows R functions to be used in this database from then on. You should be able to run the given examples in the tutorial after this is done. This was run on databases tigertest and template1.

[https://www.joeconway.com/presentations/plr-DWDC-2015.05.pdf Another possibly useful presentation on PL/R.]

==How PL/R was installed on the RDP==

On 2018-01-18, PL/R was installed to allow PostgreSQL users to use R functions from SQL queries. It was installed using [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial.] The link from the tutorial for PL/R doesn't work, I used [http://www.joeconway.com/plr.html this instead.]

It required R version 3.3.0 for PL/R to work with PostgreSQL 9.5. If the environmental variables R_HOME or PATH are edited so that they point to a different version of R, this might not work, and PL/R might have to be reinstalled. If the version of PostgreSQL is updated, both the R version and PL/R version will have to match it as well.

Using R in PostgreSQL

2018-03-06T19:15:01Z

JeeminS: /* Perhaps a Simpler Alternative */

== Perhaps a Simpler Alternative ==
Aggregate functions in PostgreSQL:
https://www.postgresql.org/docs/9.0/static/functions-aggregate.html

=== Aggregate Functions ===

Table 9-44. Aggregate Functions for Statistics

Function Argument Type Return Type Description
corr(Y, X) double precision double precision correlation coefficient
covar_pop(Y, X) double precision double precision population covariance
covar_samp(Y, X) double precision double precision sample covariance
regr_avgx(Y, X) double precision double precision average of the independent variable (sum(X)/N)
regr_avgy(Y, X) double precision double precision average of the dependent variable (sum(Y)/N)
regr_count(Y, X) double precision bigint number of input rows in which both expressions are nonnull
regr_intercept(Y, X) double precision double precision y-intercept of the least-squares-fit linear equation determined by the (X, Y) pairs
regr_r2(Y, X) double precision double precision square of the correlation coefficient
regr_slope(Y, X) double precision double precision slope of the least-squares-fit linear equation determined by the (X, Y) pairs
regr_sxx(Y, X) double precision double precision sum(X^2) - sum(X)^2/N ("sum of squares" of the independent variable)
regr_sxy(Y, X) double precision double precision sum(X*Y) - sum(X) * sum(Y)/N ("sum of products" of independent times dependent variable)
regr_syy(Y, X) double precision double precision sum(Y^2) - sum(Y)^2/N ("sum of squares" of the dependent variable)
stddev(expression) smallint, int, bigint, real, double precision, or numeric double precision for floating-point arguments, otherwise numeric historical alias for stddev_samp
stddev_pop(expression) smallint, int, bigint, real, double precision, or numeric double precision for floating-point arguments, otherwise numeric population standard deviation of the input values
stddev_samp(expression) smallint, int, bigint, real, double precision, or numeric double precision for floating-point arguments, otherwise numeric sample standard deviation of the input values
variance(expression) smallint, int, bigint, real, double precision, or numeric double precision for floating-point arguments, otherwise numeric historical alias for var_samp
var_pop(expression) smallint, int, bigint, real, double precision, or numeric double precision for floating-point arguments, otherwise numeric population variance of the input values (square of the population standard deviation)
var_samp(expression) smallint, int, bigint, real, double precision, or numeric double precision for floating-point arguments, otherwise numeric sample variance of the input values (square of the sample standard deviation)

==Joe Conway's Documentation on pl/r==

http://joeconway.com/doc/plr-US.pdf
http://www.joeconway.com/plr/doc/plr-funcs.html
http://www.joeconway.com/plr/doc/plr-window-funcs.html

=== Sample Function Code in PostgreSQL ===

==== Creating the Sample Function ====
Inside a database (ex) tigertest):

CREATE OR REPLACE FUNCTION r_max (integer, integer) RETURNS integer AS ’
if (arg1 > arg2)
return(arg1)
else
return(arg2)
’ LANGUAGE ’plr’ STRICT;

==== Calling the Sample Function ====

SELECT * FROM r_max(2,3);

returns

r_max
-------
3
(1 row)

== hcllayerwinv table from tigertest database ==

Table "public.hcllayerwinv"
Column | Type | Collation | Nullable | Default
------------------+------------------------+-----------+----------+---------
place | character varying(100) | | |
statecode | character varying(2) | | |
year | integer | | |
layer | integer | | |
lcount | numeric | | |
nosingleton | bigint | | |
nomultiton | bigint | | |
nopair | bigint | | |
nohull | bigint | | |
totmultitoncount | numeric | | |
avgmultitoncount | numeric | | |
totpaircount | numeric | | |
avgpaircount | numeric | | |
totpairlength | double precision | | |
avgpairlength | double precision | | |
minpairlength | double precision | | |
maxpairlength | double precision | | |
tothullcount | numeric | | |
avghullcount | numeric | | |
tothullarea | double precision | | |
avghullarea | double precision | | |
minhullarea | double precision | | |
maxhullarea | double precision | | |
tothulldensity | double precision | | |
avghulldensity | double precision | | |
seedearlyinvf | double precision | | |
inflator | numeric | | |
seedearlyinvl16f | double precision | | |

=== Regress by===
--By: place, statecode, layer,
--Regress: seedearlyinvl16f ON nosingleton, totmultitoncount, totpaircount, tothullcount, avgpairlength, avghullarea, avghulldensity

==Installing PL/R on the dbase server==

Check the version of the dbase by doing a
SELECT version();

This gives us 9.5.10

Then as root:
apt-get install postgresql-9.5-plr

Finally enable the extension in a dbase:
create extension plr;

See also: http://www.joeconway.com/presentations/plr-PGConfNYC2014.pdf

==Instructions on Use==

To use R from pgAdmin III, follow the instructions in [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial]: choose the database, click SQL, and run "CREATE EXTENSION plr;". This allows R functions to be used in this database from then on. You should be able to run the given examples in the tutorial after this is done. This was run on databases tigertest and template1.

[https://www.joeconway.com/presentations/plr-DWDC-2015.05.pdf Another possibly useful presentation on PL/R.]

==How PL/R was installed on the RDP==

On 2018-01-18, PL/R was installed to allow PostgreSQL users to use R functions from SQL queries. It was installed using [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial.] The link from the tutorial for PL/R doesn't work, I used [http://www.joeconway.com/plr.html this instead.]

It required R version 3.3.0 for PL/R to work with PostgreSQL 9.5. If the environmental variables R_HOME or PATH are edited so that they point to a different version of R, this might not work, and PL/R might have to be reinstalled. If the version of PostgreSQL is updated, both the R version and PL/R version will have to match it as well.

Using R in PostgreSQL

2018-03-02T18:25:28Z

JeeminS:

== Perhaps a Simpler Alternative ==
Aggregate functions in PostgreSQL:
https://www.postgresql.org/docs/9.0/static/functions-aggregate.html

==Joe Conway's Documentation on pl/r==

http://joeconway.com/doc/plr-US.pdf
http://www.joeconway.com/plr/doc/plr-funcs.html
http://www.joeconway.com/plr/doc/plr-window-funcs.html

=== Sample Function Code in PostgreSQL ===

==== Creating the Sample Function ====
Inside a database (ex) tigertest):

CREATE OR REPLACE FUNCTION r_max (integer, integer) RETURNS integer AS ’
if (arg1 > arg2)
return(arg1)
else
return(arg2)
’ LANGUAGE ’plr’ STRICT;

==== Calling the Sample Function ====

SELECT * FROM r_max(2,3);

returns

r_max
-------
3
(1 row)

== hcllayerwinv table from tigertest database ==

Table "public.hcllayerwinv"
Column | Type | Collation | Nullable | Default
------------------+------------------------+-----------+----------+---------
place | character varying(100) | | |
statecode | character varying(2) | | |
year | integer | | |
layer | integer | | |
lcount | numeric | | |
nosingleton | bigint | | |
nomultiton | bigint | | |
nopair | bigint | | |
nohull | bigint | | |
totmultitoncount | numeric | | |
avgmultitoncount | numeric | | |
totpaircount | numeric | | |
avgpaircount | numeric | | |
totpairlength | double precision | | |
avgpairlength | double precision | | |
minpairlength | double precision | | |
maxpairlength | double precision | | |
tothullcount | numeric | | |
avghullcount | numeric | | |
tothullarea | double precision | | |
avghullarea | double precision | | |
minhullarea | double precision | | |
maxhullarea | double precision | | |
tothulldensity | double precision | | |
avghulldensity | double precision | | |
seedearlyinvf | double precision | | |
inflator | numeric | | |
seedearlyinvl16f | double precision | | |

=== Regress by===
--By: place, statecode, layer,
--Regress: seedearlyinvl16f ON nosingleton, totmultitoncount, totpaircount, tothullcount, avgpairlength, avghullarea, avghulldensity

==Installing PL/R on the dbase server==

Check the version of the dbase by doing a
SELECT version();

This gives us 9.5.10

Then as root:
apt-get install postgresql-9.5-plr

Finally enable the extension in a dbase:
create extension plr;

See also: http://www.joeconway.com/presentations/plr-PGConfNYC2014.pdf

==Instructions on Use==

To use R from pgAdmin III, follow the instructions in [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial]: choose the database, click SQL, and run "CREATE EXTENSION plr;". This allows R functions to be used in this database from then on. You should be able to run the given examples in the tutorial after this is done. This was run on databases tigertest and template1.

[https://www.joeconway.com/presentations/plr-DWDC-2015.05.pdf Another possibly useful presentation on PL/R.]

==How PL/R was installed on the RDP==

On 2018-01-18, PL/R was installed to allow PostgreSQL users to use R functions from SQL queries. It was installed using [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial.] The link from the tutorial for PL/R doesn't work, I used [http://www.joeconway.com/plr.html this instead.]

It required R version 3.3.0 for PL/R to work with PostgreSQL 9.5. If the environmental variables R_HOME or PATH are edited so that they point to a different version of R, this might not work, and PL/R might have to be reinstalled. If the version of PostgreSQL is updated, both the R version and PL/R version will have to match it as well.

Using R in PostgreSQL

2018-03-01T20:01:13Z

JeeminS: /* Joe Conway's Documentation on pl/r */

==Joe Conway's Documentation on pl/r==

http://joeconway.com/doc/plr-US.pdf
http://www.joeconway.com/plr/doc/plr-funcs.html
http://www.joeconway.com/plr/doc/plr-window-funcs.html

=== Sample Function Code in PostgreSQL ===

==== Creating the Sample Function ====
Inside a database (ex) tigertest):

CREATE OR REPLACE FUNCTION r_max (integer, integer) RETURNS integer AS ’
if (arg1 > arg2)
return(arg1)
else
return(arg2)
’ LANGUAGE ’plr’ STRICT;

==== Calling the Sample Function ====

SELECT * FROM r_max(2,3);

returns

r_max
-------
3
(1 row)

== hcllayerwinv table from tigertest database ==

Table "public.hcllayerwinv"
Column | Type | Collation | Nullable | Default
------------------+------------------------+-----------+----------+---------
place | character varying(100) | | |
statecode | character varying(2) | | |
year | integer | | |
layer | integer | | |
lcount | numeric | | |
nosingleton | bigint | | |
nomultiton | bigint | | |
nopair | bigint | | |
nohull | bigint | | |
totmultitoncount | numeric | | |
avgmultitoncount | numeric | | |
totpaircount | numeric | | |
avgpaircount | numeric | | |
totpairlength | double precision | | |
avgpairlength | double precision | | |
minpairlength | double precision | | |
maxpairlength | double precision | | |
tothullcount | numeric | | |
avghullcount | numeric | | |
tothullarea | double precision | | |
avghullarea | double precision | | |
minhullarea | double precision | | |
maxhullarea | double precision | | |
tothulldensity | double precision | | |
avghulldensity | double precision | | |
seedearlyinvf | double precision | | |
inflator | numeric | | |
seedearlyinvl16f | double precision | | |

=== Regress by===
--By: place, statecode, layer,
--Regress: seedearlyinvl16f ON nosingleton, totmultitoncount, totpaircount, tothullcount, avgpairlength, avghullarea, avghulldensity

==Installing PL/R on the dbase server==

Check the version of the dbase by doing a
SELECT version();

This gives us 9.5.10

Then as root:
apt-get install postgresql-9.5-plr

Finally enable the extension in a dbase:
create extension plr;

See also: http://www.joeconway.com/presentations/plr-PGConfNYC2014.pdf

==Instructions on Use==

To use R from pgAdmin III, follow the instructions in [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial]: choose the database, click SQL, and run "CREATE EXTENSION plr;". This allows R functions to be used in this database from then on. You should be able to run the given examples in the tutorial after this is done. This was run on databases tigertest and template1.

[https://www.joeconway.com/presentations/plr-DWDC-2015.05.pdf Another possibly useful presentation on PL/R.]

==How PL/R was installed on the RDP==

On 2018-01-18, PL/R was installed to allow PostgreSQL users to use R functions from SQL queries. It was installed using [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial.] The link from the tutorial for PL/R doesn't work, I used [http://www.joeconway.com/plr.html this instead.]

It required R version 3.3.0 for PL/R to work with PostgreSQL 9.5. If the environmental variables R_HOME or PATH are edited so that they point to a different version of R, this might not work, and PL/R might have to be reinstalled. If the version of PostgreSQL is updated, both the R version and PL/R version will have to match it as well.

Using R in PostgreSQL

2018-02-27T18:53:02Z

JeeminS: /* hcllayerwinv table from tigertest database */

==Joe Conway's Documentation on pl/r==

http://joeconway.com/doc/plr-US.pdf

=== Sample Function Code in PostgreSQL ===

==== Creating the Sample Function ====
Inside a database (ex) tigertest):

CREATE OR REPLACE FUNCTION r_max (integer, integer) RETURNS integer AS ’
if (arg1 > arg2)
return(arg1)
else
return(arg2)
’ LANGUAGE ’plr’ STRICT;

==== Calling the Sample Function ====

SELECT * FROM r_max(2,3);

returns

r_max
-------
3
(1 row)

== hcllayerwinv table from tigertest database ==

Table "public.hcllayerwinv"
Column | Type | Collation | Nullable | Default
------------------+------------------------+-----------+----------+---------
place | character varying(100) | | |
statecode | character varying(2) | | |
year | integer | | |
layer | integer | | |
lcount | numeric | | |
nosingleton | bigint | | |
nomultiton | bigint | | |
nopair | bigint | | |
nohull | bigint | | |
totmultitoncount | numeric | | |
avgmultitoncount | numeric | | |
totpaircount | numeric | | |
avgpaircount | numeric | | |
totpairlength | double precision | | |
avgpairlength | double precision | | |
minpairlength | double precision | | |
maxpairlength | double precision | | |
tothullcount | numeric | | |
avghullcount | numeric | | |
tothullarea | double precision | | |
avghullarea | double precision | | |
minhullarea | double precision | | |
maxhullarea | double precision | | |
tothulldensity | double precision | | |
avghulldensity | double precision | | |
seedearlyinvf | double precision | | |
inflator | numeric | | |
seedearlyinvl16f | double precision | | |

=== Regress by===
--By: place, statecode, layer,
--Regress: seedearlyinvl16f ON nosingleton, totmultitoncount, totpaircount, tothullcount, avgpairlength, avghullarea, avghulldensity

==Installing PL/R on the dbase server==

Check the version of the dbase by doing a
SELECT version();

This gives us 9.5.10

Then as root:
apt-get install postgresql-9.5-plr

Finally enable the extension in a dbase:
create extension plr;

See also: http://www.joeconway.com/presentations/plr-PGConfNYC2014.pdf

==Instructions on Use==

To use R from pgAdmin III, follow the instructions in [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial]: choose the database, click SQL, and run "CREATE EXTENSION plr;". This allows R functions to be used in this database from then on. You should be able to run the given examples in the tutorial after this is done. This was run on databases tigertest and template1.

[https://www.joeconway.com/presentations/plr-DWDC-2015.05.pdf Another possibly useful presentation on PL/R.]

==How PL/R was installed on the RDP==

On 2018-01-18, PL/R was installed to allow PostgreSQL users to use R functions from SQL queries. It was installed using [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial.] The link from the tutorial for PL/R doesn't work, I used [http://www.joeconway.com/plr.html this instead.]

It required R version 3.3.0 for PL/R to work with PostgreSQL 9.5. If the environmental variables R_HOME or PATH are edited so that they point to a different version of R, this might not work, and PL/R might have to be reinstalled. If the version of PostgreSQL is updated, both the R version and PL/R version will have to match it as well.

Using R in PostgreSQL

2018-02-27T18:43:34Z

JeeminS:

==Joe Conway's Documentation on pl/r==

http://joeconway.com/doc/plr-US.pdf

=== Sample Function Code in PostgreSQL ===

==== Creating the Sample Function ====
Inside a database (ex) tigertest):

CREATE OR REPLACE FUNCTION r_max (integer, integer) RETURNS integer AS ’
if (arg1 > arg2)
return(arg1)
else
return(arg2)
’ LANGUAGE ’plr’ STRICT;

==== Calling the Sample Function ====

SELECT * FROM r_max(2,3);

returns

r_max
-------
3
(1 row)

== hcllayerwinv table from tigertest database ==

Table "public.hcllayerwinv"
Column | Type | Collation | Nullable | Default
------------------+------------------------+-----------+----------+---------
place | character varying(100) | | |
statecode | character varying(2) | | |
year | integer | | |
layer | integer | | |
lcount | numeric | | |
nosingleton | bigint | | |
nomultiton | bigint | | |
nopair | bigint | | |
nohull | bigint | | |
totmultitoncount | numeric | | |
avgmultitoncount | numeric | | |
totpaircount | numeric | | |
avgpaircount | numeric | | |
totpairlength | double precision | | |
avgpairlength | double precision | | |
minpairlength | double precision | | |
maxpairlength | double precision | | |
tothullcount | numeric | | |
avghullcount | numeric | | |
tothullarea | double precision | | |
avghullarea | double precision | | |
minhullarea | double precision | | |
maxhullarea | double precision | | |
tothulldensity | double precision | | |
avghulldensity | double precision | | |
seedearlyinvf | double precision | | |
inflator | numeric | | |
seedearlyinvl16f | double precision | | |

==Installing PL/R on the dbase server==

Check the version of the dbase by doing a
SELECT version();

This gives us 9.5.10

Then as root:
apt-get install postgresql-9.5-plr

Finally enable the extension in a dbase:
create extension plr;

See also: http://www.joeconway.com/presentations/plr-PGConfNYC2014.pdf

==Instructions on Use==

To use R from pgAdmin III, follow the instructions in [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial]: choose the database, click SQL, and run "CREATE EXTENSION plr;". This allows R functions to be used in this database from then on. You should be able to run the given examples in the tutorial after this is done. This was run on databases tigertest and template1.

[https://www.joeconway.com/presentations/plr-DWDC-2015.05.pdf Another possibly useful presentation on PL/R.]

==How PL/R was installed on the RDP==

On 2018-01-18, PL/R was installed to allow PostgreSQL users to use R functions from SQL queries. It was installed using [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial.] The link from the tutorial for PL/R doesn't work, I used [http://www.joeconway.com/plr.html this instead.]

It required R version 3.3.0 for PL/R to work with PostgreSQL 9.5. If the environmental variables R_HOME or PATH are edited so that they point to a different version of R, this might not work, and PL/R might have to be reinstalled. If the version of PostgreSQL is updated, both the R version and PL/R version will have to match it as well.

Using R in PostgreSQL

2018-02-22T19:43:05Z

JeeminS: /* Joe Conway's Documentation on pl/r */

Using R in PostgreSQL

2018-02-22T19:42:40Z

JeeminS: /* Sample Function Code in PostgreSQL */

Using R in PostgreSQL

2018-02-22T19:41:37Z

JeeminS:

==Joe Conway's Documentation on pl/r==

http://joeconway.com/doc/plr-US.pdf

=== Sample Function Code in PostgreSQL ===

Inside a database (ex) tigertest):

CREATE OR REPLACE FUNCTION r_max (integer, integer) RETURNS integer AS ’
if (arg1 > arg2)
return(arg1)
else
return(arg2)
’ LANGUAGE ’plr’ STRICT;

==Installing PL/R on the dbase server==

Check the version of the dbase by doing a
SELECT version();

This gives us 9.5.10

Then as root:
apt-get install postgresql-9.5-plr

Finally enable the extension in a dbase:
create extension plr;

See also: http://www.joeconway.com/presentations/plr-PGConfNYC2014.pdf

==Instructions on Use==

To use R from pgAdmin III, follow the instructions in [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial]: choose the database, click SQL, and run "CREATE EXTENSION plr;". This allows R functions to be used in this database from then on. You should be able to run the given examples in the tutorial after this is done. This was run on databases tigertest and template1.

[https://www.joeconway.com/presentations/plr-DWDC-2015.05.pdf Another possibly useful presentation on PL/R.]

==How PL/R was installed on the RDP==

On 2018-01-18, PL/R was installed to allow PostgreSQL users to use R functions from SQL queries. It was installed using [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial.] The link from the tutorial for PL/R doesn't work, I used [http://www.joeconway.com/plr.html this instead.]

It required R version 3.3.0 for PL/R to work with PostgreSQL 9.5. If the environmental variables R_HOME or PATH are edited so that they point to a different version of R, this might not work, and PL/R might have to be reinstalled. If the version of PostgreSQL is updated, both the R version and PL/R version will have to match it as well.

Using R in PostgreSQL

2018-02-22T19:39:28Z

JeeminS:

==Joe Conway's Documentation on pl/r==

http://joeconway.com/doc/plr-US.pdf

==Installing PL/R on the dbase server==

Check the version of the dbase by doing a
SELECT version();

This gives us 9.5.10

Then as root:
apt-get install postgresql-9.5-plr

Finally enable the extension in a dbase:
create extension plr;

See also: http://www.joeconway.com/presentations/plr-PGConfNYC2014.pdf

==Instructions on Use==

To use R from pgAdmin III, follow the instructions in [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial]: choose the database, click SQL, and run "CREATE EXTENSION plr;". This allows R functions to be used in this database from then on. You should be able to run the given examples in the tutorial after this is done. This was run on databases tigertest and template1.

[https://www.joeconway.com/presentations/plr-DWDC-2015.05.pdf Another possibly useful presentation on PL/R.]

==How PL/R was installed on the RDP==

On 2018-01-18, PL/R was installed to allow PostgreSQL users to use R functions from SQL queries. It was installed using [https://www.bostongis.com/PrinterFriendly.aspx?content_name=postgresql_plr_tut01 this tutorial.] The link from the tutorial for PL/R doesn't work, I used [http://www.joeconway.com/plr.html this instead.]

It required R version 3.3.0 for PL/R to work with PostgreSQL 9.5. If the environmental variables R_HOME or PATH are edited so that they point to a different version of R, this might not work, and PL/R might have to be reinstalled. If the version of PostgreSQL is updated, both the R version and PL/R version will have to match it as well.

Urban Start-up Agglomeration and Venture Capital Investment

2018-01-11T20:10:23Z

JeeminS:

{{AcademicPaper
|Has title=Urban Start-up Agglomeration and Venture Capital Investment
|Has author=Ed Egan,
|Has RAs=Peter Jalbert, Jake Silberman, Christy Warden, Jeemin Sim,
|Has paper status=In development
}}
==Summary==

Agglomeration is generally thought to be one of the most important determinants of growth for urban entrepreneurship ecosystems. However, there is essentially no empirical evidence to support this. This paper takes advantage of geocoding and introduces a novel measure of agglomeration. This measure is the smallest circle area that covers all startup offices, subject to having at least N startups in each circle. Using GIS data on cities, this paper controls for the density and socio-demographics of an area to identify the effect of just agglomeration.

==Description==

Clusters of economic activity plays a significant role in the firms performance and growth. An important driver of growth is the knowledge spillover between firms. This includes among others the facilitation of information flow and ideas between firms which could be a milestone especially in the growth of startup firms or small businesses. This project focuses on the effects of agglomeration on the performance and growth of startup firms. It introduces a novel measure of agglomeration which can be used to empirically test the effects of clustering. This measure the is smallest total circle area that covers all of the startups in the sample such that there are at least n firms in each circle. The projects is based on the creation of an algorithm which gives an unbiased measure to be used in the empirical analysis. The regression we are interested in takes the following form:

[[File:regression_equation.png]]

The dependent variable is a measure of growth of the firms. This measure could be investment forwarded one period or growth in investment. The control variables include the number of the startups firms, m, the agglomeration measure, A and a vector of other control variables affecting the growth of firms at time t. Because of the endogeneity in the circle area or the measure of agglomeration, A, there is a need for an instrumental variable to get consistent estimates of the effects we are interested in. The proposed instrument is the presence of a river, or road in between the points representing geographical locations of the venture capital backed up firms. The instrument affects agglomeration without having a direct impact on the growth. This makes it good candidate for a valid instrument.
The next tasks are determining the additional control variables to include in the regression, years to include in the analysis and methods of finding an unbiased measure of agglomeration.

==Data==

===Making the circle input data===

Ed's additional datawork is in
Z:\VentureCapitalData\SDCVCData\vcdb2\ProcessingCoLevelSimple.sql

The key table for circle processing is CoLevelBlowout, which is restricted (to include cities with greater than 10 active at some point in the data) to make CoLevelForCircles.

We need to:
#Winsorize CoLevelBlowout
#Compute the circles!
#Make the Bay Area (over time) data
#Plot the Bay Area data (with colors per Bay Area city) for 1985 to present
#Combine the plots to make an animated gif

To winsorize the data we need the formula for [https://en.wikipedia.org/wiki/Great-circle_distance Great Circle Distance]. The radius of the earth is 6,378km (half of diameter: 12,756 km). So:
GCD = acos( sin(lat1) x sin(lat2) + cos(lat1) x cos(lat2) x cos(long1-long2) ) x r

===Main Sources===

The primary sources of data for this project are:
*SDC VentureXpert - from [[VC Database Rebuild]], the key table is '''
*GIS City Data
*Data on NSF, NIH, population, income, clinical trials, employment, schooling, R&D expenditures and revenue of firms can be found in [[Hubs]].

===VC data===

Data on the number of new vc backed firms in each city and year is in:
Z:\Hubs\2017\clean data
The name of the file is '''firm_nr.txt'''.
Database is cities
SQL script is: '''nr_firms.sql'''

Raw data is in:
Z:\VentureCapitalData\SDCVCData\vcdb2
The file is '''colevelsimple.txt'''

In order to see if there are outliers, I get the average coordinates for all cities and find the differences of the firm's coordinates from the city coordinate.
The script for the average city coordinates is in
Z:\Hubs\2017\sql scripts and the file name is '''newcolevel.sql'''.

The differences are taken in excel. The file containing the differences is in
Z:\Hubs\2017 and the file name is '''new_colevel.txt'''.

*Data on the circle area in each city and year is in:
Z:\Hubs\2017\clean data
The name of the file is '''circles.txt'''. (It contains only 106 observations)

Database is cities
SQL script is: '''circles.sql'''

The script for joining the two tables on the VC table is in:
Z:\Hubs\2017\sql scripts
The name of the file is '''new_firm_nr_circles.sql'''

*We use the cities with greater than 10 active VC backed firms. Data on the cities and number of active firms is in:
E:\McNair\Projects\Hubs\Summer 2017
The file is '''CitiesWithGT10Active.txt'''

The script for joining the final data with this file is located in
Z:\Hubs\2017\sql scripts
The file name is '''final_joined_kerda.sql'''.
The final data is in
Z:\Hubs\2017\clean data
The file name is '''new_final_kerda.txt'''.

===Accelerator data===

Accelerators data is in
Z:\Hubs\2017\clean data
The file name is accelerators.txt
The table is '''accelerators'''
The joined accelerators data with the VC table is in joined_accelerators table.
The script is in
Z:\Hubs\2017\sql scripts
The file name is '''join_accelerators.sql'''

The do file is in
Z:\Hubs\2017\kerda
The name is '''agglomeartion_kerda.do'''
It includes the graphs, tables and the preliminary FE regressions with VC funding amount and growth rate.
It also predicts the hazard rates, matches on the hazard rate in order to create synthetic control and treatment groups.
What is left to do is to add 2 lagged and 3 forward observations for the cities which do have a match. Remove the overlapping observations for the years that get a treatment but which at the same time serve as a control.

===See also===

Also:
*[[Enclosing Circle Algorithm]]
*Normalizer
*Geocode.py

==Unbiased measure==

The number of startups affects the total area of the circles according to some function. The task is to find an unbiased measure of the area, which is not affected by the number of the startups, given the size and their distribution.

For the unbiased calculation of a measure in a different context see: http://users.nber.org/~edegan/w/images/d/d0/Hall_(2005)_-_A_Note_On_The_Bias_In_Herfindahl_Type_Measures_Based_On_Count_Data.pdf

==GIS Resources==

*https://www.census.gov/geo/maps-data/data/tiger-line.html
*https://www.census.gov/geo/maps-data/data/tiger.html
*http://postgis.net/features/
*https://en.wikipedia.org/wiki/GIS_file_formats

===Useful functions for spatial joins===

'''sum(expression)''': aggregate to return a sum for a set of records
'''count(expression)''': aggregate to return the size of a set of records
'''ST_Area(geometry)''' returns the area of the polygons
'''ST_AsText(geometry)''' returns WKT text
'''ST_Buffer(geometry, distance)''': For geometry: Returns a geometry that represents all points whose distance from this Geometry is less than or equal to distance. Calculations are in the Spatial Reference System of this Geometry. For geography: Uses a planar transform wrapper.
'''ST_Contains(geometry A, geometry B)''' returns the true if geometry A contains geometry B
'''ST_Distance(geometry A, geometry B)''' returns the minimum distance between geometry A and geometry B
'''ST_DWithin(geometry A, geometry B, radius)''' returns the true if geometry A is radius distance or less from geometry B
'''ST_GeomFromText(text)''' returns geometry
'''ST_Intersection(geometry A, geometry B)''': Returns a geometry that represents the shared portion of geomA and geomB. The geography implementation does a transform to geometry to do the intersection and then transform back to WGS84
'''ST_Intersects(geometry A, geometry B)''' returns the true if geometry A intersects geometry B
'''ST_Length(linestring)''' returns the length of the linestring
'''ST_Touches(geometry A, geometry B)''' returns the true if the boundary of geometry A touches geometry B
'''ST_Within(geometry A, geometry B)''' returns the true if geometry A is within geometry B
geometry_a '''&&''' geometry_b: Returns TRUE if A’s bounding box overlaps B’s.
geometry_a '''=''' geometry_b: Returns TRUE if A’s bounding box is the same as B’s.
'''ST_SetSRID(geometry, srid)''': Sets the SRID on a geometry to a particular integer value.
'''ST_SRID(geometry)''': Returns the spatial reference identifier for the ST_Geometry as defined in spatial_ref_sys table.
'''ST_Transform(geometry, srid)''': Returns a new geometry with its coordinates transformed to the SRID referenced by the integer parameter.
'''ST_Union()''': Returns a geometry that represents the point set union of the Geometries.
'''substring(string [from int] [for int])''': PostgreSQL string function to extract substring matching SQL regular expression.
'''ST_Relate(geometry A, geometry B)''': Returns a text string representing the DE9IM relationship between the geometries.
'''ST_GeoHash(geometry A)''': Returns a text string representing the GeoHash of the bounds of the object.

===Native functions for geography===

'''ST_AsText(geography)''' returns text
'''ST_GeographyFromText(text)''' returns geography
'''ST_AsBinary(geography)''' returns bytea
'''ST_GeogFromWKB(bytea)''' returns geography
'''ST_AsSVG(geography)''' returns text
'''ST_AsGML(geography)''' returns text
'''ST_AsKML(geography)''' returns text
'''ST_AsGeoJson(geography)''' returns text
'''ST_Distance(geography, geography)''' returns double
'''ST_DWithin(geography, geography, float8)''' returns boolean
'''ST_Area(geography)''' returns double
'''ST_Length(geography)''' returns double
'''ST_Covers(geography, geography)''' returns boolean
'''ST_CoveredBy(geography, geography)''' returns boolean
'''ST_Intersects(geography, geography)''' returns boolean
'''ST_Buffer(geography, float8)''' returns geography [1]
'''ST_Intersection(geography, geography)''' returns geography [1]

===Functions for Linear Referencing===
'''ST_LineInterpolatePoint(geometry A, double measure)''': Returns a point interpolated along a line.
'''ST_LineLocatePoint(geometry A, geometry B)''': Returns a float between 0 and 1 representing the location of the closest point on LineString to the given Point.
'''ST_Line_Substring(geometry A, double from, double to)''': Return a linestring being a substring of the input one starting and ending at the given fractions of total 2d length.
'''ST_Locate_Along_Measure(geometry A, double measure)''': Return a derived geometry collection value with elements that match the specified measure.
'''ST_Locate_Between_Measures(geometry A, double from, double to)''': Return a derived geometry collection value with elements that match the specified range of measures inclusively.
'''ST_AddMeasure(geometry A, double from, double to)''': Return a derived geometry with measure elements linearly interpolated between the start and end points. If the geometry has no measure dimension, one is added.

===3-D Functions===
'''ST_3DClosestPoint''' — Returns the 3-dimensional point on g1 that is closest to g2. This is the first point of the 3D shortest line.
'''ST_3DDistance''' — For geometry type Returns the 3-dimensional cartesian minimum distance (based on spatial ref) between two geometries in projected units.
'''ST_3DDWithin''' — For 3d (z) geometry type Returns true if two geometries 3d distance is within number of units.
'''ST_3DDFullyWithin''' — Returns true if all of the 3D geometries are within the specified distance of one another.
'''ST_3DIntersects''' — Returns TRUE if the Geometries “spatially intersect” in 3d - only for points and linestrings
'''ST_3DLongestLine''' — Returns the 3-dimensional longest line between two geometries
'''ST_3DMaxDistance''' — For geometry type Returns the 3-dimensional cartesian maximum distance (based on spatial ref) between two geometries in projected units.
'''ST_3DShortestLine''' — Returns the 3-dimensional shortest line between two geometries

===Relevant PostgreSQL Commands===
'''\dt *.*''' Show all tables
'''\q''' Exit table

===Specifities/ Outliers to consider===
New York (decompose)
Princeton area (keep Princeton unique)
Reston, Virginia (keep)
San Diego (include La Jolla)
Silicon Valley (all distinct)

===To make a circle===

SELECT ST_Buffer(''[desired point]'', ''[desired radius]'', 'quad_segs=8')
FROM ''[desired table]''
quad_segs=8 indicates circle

[[File: CirclePostGIS.png]]

For more precision in circle:
SELECT ST_Transform(geometry(
ST_Buffer(geography(
ST_Transform( ''[desired point]'', 4326 )),
''[desired radius]')),
900913) FROM ''[desired table]''
4326 and 900913 represent particular precision.

===Decimal Degrees===

We are working with longitude and latitude in decimal degrees. See https://en.wikipedia.org/wiki/Decimal_degrees

When converting radius to km, multiply by 111.3199. For area, multiple by (111.3199)^2=12,392.12013601.

==Census Data==

===Population===

The Census Gazetteer files for 2010, 2000 and 1990 can give use population by census place. See https://www.census.gov/geo/maps-data/data/gazetteer.html

The places file contains data for all incorporated places and census designated places (CDPs) in the 50 states, the District of Columbia and Puerto Rico as of the January 1, 2010. The file is tab-delimited text, one line per record. Some records contain special characters.
Download the National Places Gazetteer Files (1.2MB)
Download the State-Based Places Gazetteer Files:
Column Label Description
Column 1 USPS United States Postal Service State Abbreviation
Column 2 GEOID Geographic Identifier - fully concatenated geographic code (State FIPS and Place FIPS)
Column 3 ANSICODE American National Standards Insititute code
Column 4 NAME Name
Column 5 LSAD Legal/Statistical area descriptor.
Column 6 FUNCSTAT Functional status of entity.
Column 7 POP10 2010 Census population count.
Column 8 HU10 2010 Census housing unit count.
Column 9 ALAND Land Area (square meters) - Created for statistical purposes only.
Column 10 AWATER Water Area (square meters) - Created for statistical purposes only.
Column 11 ALAND_SQMI Land Area (square miles) - Created for statistical purposes only.
Column 12 AWATER_SQMI Water Area (square miles) - Created for statistical purposes only.
Column 13 INTPTLAT Latitude (decimal degrees) First character is blank or "-" denoting North or South latitude respectively
Column 14 INTPTLONG Longitude (decimal degrees) First character is blank or "-" denoting East or West longitude respectively.

===Relationships===

See https://www.census.gov/geo/maps-data/data/relationship.html

These text files describe geographic relationships. There are two types of relationship files; those that show the relationship between the same type of geography over time (comparability) and those that show the relationship between two types of geography for the same time period.

===ACS (American Community Survey) Data ===

Steps to download:

1) Go to https://factfinder.census.gov/faces/nav/jsf/pages/download_center.xhtml
2) Select 'I know the dataset or table(s) that I want to download.'
3) Press Next
4) For 'Select a program:' choose
'American Community Survey'
5) For 'Select a dataset and click Add to Your Selections:' choose
'<YEAR OF INTEREST> ACS 1-year estimates'
6) Press 'Add To Your Selections'
7) Press Next
8) For 'Select a geographic type:' choose
'Place - 160'
9) For Select a state:
Don't choose a state, as we wish to download all.
10) For 'Select one or more geographic areas...' choose
'All Places within United States and Puerto Rico'
11) Press Next

===Other===

Counts of firms by NAICS code at the county level may be useful: https://www2.census.gov/geo/pdfs/education/cbp12gdbs.pdf

==Tax Increment Finance Zones==

*State by state enabling statues: https://www.cdfa.net/cdfa/tifmap.nsf/index.html

Jeemin Sim (Work Log)

2018-01-11T20:10:06Z

JeeminS:

[[Jeemin Sim]] [[Work Logs]] [[Jeemin Sim (WorkLog)|(log page)]]

===Spring 2018===
2017-01-11:
*Was briefed on [[Urban Start-up Agglomeration and Venture Capital Investment]]
*Going forward - running regressions in R in PostgreSQL and with desired layers (instead of levels)

===Fall 2017===
<onlyinclude>

2017-12-05:
* Uploaded tables with KML files in TIF folder in bulk(E:) drive
** Allegheny County
** Atlanta
** Chicago
** Columbus, OH
** Dublin, OH
** Vermont
** Washington, D.C.

*Updated documentation for:
** http://mcnair.bakerinstitute.org/wiki/TIF_Project#Uploading_TIF_Data_onto_database_.28tigertest.29

* shp2pgsql needs to be installed to upload Shapefiles to PostgreSQL database
** applies for Houston, TX and Dallas, TX
** DONE

2017-12-04:
* To upload KML file into database with specified table name:
researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
* chicagotif table now resides in tigertest database

2017-11-30:
* Displayed map with TIF districts and startups
* Location:
E:\McNair\Projects\Agglomeration\TIF\McNair_Color_Chicago_TIF_and_startups.png
* Added information and steps to ArcMap/ArcGIS documentation page
* Create a project page for 'Working with POSTGIS' and add instructions for uploading KML file onto POSTGIS
** Command used :
*** Logged in as : researcher@McNairDBServ
/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml
* chicago TIF kml file currently downloaded in tigertest with a table name of Layer0
** Figure out how to change layer name while loading kml file
** Instructions pulled from : http://wiki.wildsong.biz/index.php/Loading_data_into_PostGIS#Loading_data_from_KMZ_files

2017-11-28:
* Created and edited [[ArcMap / ArcGIS Documentation]]
* Plotted points for TIFS and Startups in Chicago in one map.
** Location:
E:\McNair\Projects\Agglomeration\TIF\Jeemin_Chicago_TIF_and_Startups_Attempt1
* Used https://mygeodata.cloud/result to convert from KML file to CSV (which was then saved as txt file to be uploaded onto ArcMap)
* Text files located in Local Disk (C:) Drive

2017-11-27:
*Download Chicago TIF data

2017-11-13:
*

2017-11-09:
* Notes on data downloaded:
** Year 2010-2012 data are based on total population, not 25 yrs or over (case for all other tables)
*** Record appears five times total, with same exact column name
***For exmaple: 'Total; Estimate; High school graduate (includes equivalency)' appears five times, with different values.
* TODO:
** Make Projects page for ACS Data
***[[American Community Survey (ACS) Data]]

2017-11-07:
*Yesterday, narrowed down columns of interest from ACS_S1501_educationattain_2016 table.

Id
Id2
Geography
Total; Estimate; Population 25 years and over
Total; Estimate; Population 25 years and over - High school graduate (includes equivalency)
Total; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Total; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Percent; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Total; Estimate; Population 25 years and over - Associate's degree
Total; Margin of Error; Population 25 years and over - Associate's degree
Percent; Estimate; Population 25 years and over - Associate's degree
Percent; Margin of Error; Population 25 years and over - Associate's degree
Total; Estimate; Population 25 years and over - Bachelor's degree
Total; Margin of Error; Population 25 years and over - Bachelor's degree
Percent; Estimate; Population 25 years and over - Bachelor's degree
Percent; Margin of Error; Population 25 years and over - Bachelor's degree
Total; Estimate; Population 25 years and over - Graduate or professional degree
Total; Margin of Error; Population 25 years and over - Graduate or professional degree
Percent; Estimate; Population 25 years and over - Graduate or professional degree
Percent; Margin of Error; Population 25 years and over - Graduate or professional degree
Percent; Estimate; Percent high school graduate or higher
Percent; Margin of Error; Percent high school graduate or higher
Percent; Estimate; Percent bachelor's degree or higher
Percent; Margin of Error; Percent bachelor's degree or higher

*Complications:
**For csv files corresponding to years 2015 & 2016, all of the above columns exist.
**For csv files corresponding to years 2005 - 2014, no 'Percent' columns exist
*** Instead their 'Total' columns are percentage values
**For csv file corresponding to year 2005, columns regarding Graduate or professional degree are labeled differently.
**2012 data doesn't correspond to Population 25 years and over.

*Temporary Solution:
**Since the above problems may be specific to this set of tables, will go through csv files and adjust columns.

*Python script location:
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data\pullCertainColumns.py

2017-10-31:
* Finished doanloading files from ACS.
* Started loading tables into tigertest.
* Commands run could be found in
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data\DataLoading_SQL_Commands.txt

2017-10-30:
* Downloaded data from ACS, to be continued
* File path:
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data
* Fields of interest:
S1401 SCHOOL ENROLLMENT
S1501 EDUCATIONAL ATTAINMENT
S2301 EMPLOYMENT STATUS
B01003 TOTAL POPULATION
B02001 RACE
B07201 GEOGRAPHICAL MOBILITY
B08303 TRAVEL TIME TO WORK
B19013 MEDIAN HOUSEHOLD INCOME
B19053 SELF-EMPLOYMENT INCOME IN THE PAST 12 MONTHS FOR HOUSEHOLDS
B19083 GINI INDEX OF INCOME INEQUALITY
B25003 TENURE
B25105 MEDIAN MONTHLY HOUSING COSTS
B28011 INTERNET SUBSCRIPTIONS IN HOUSEHOLD
G001 GEOGRAPHIC IDENTIFIERS

2017-10-23:
* Talked to Ed with Peter & Oliver about upcoming tasks & projects.
* Loaded acs_place table 2017 (does not contain population) on tigertest.
** SQL commands used:

DROP TABLE acs_place;

CREATE TABLE acs_place (
USPS varchar(5),
GEOID varchar(30),
ANSICODE varchar(30),
NAME varchar(100),
LSAD varchar(30),
FUNCSTAT varchar(10),
ALAND varchar(30),
AWATER varchar(30),
ALAND_SQMI varchar(30),
AWATER_SQMI varchar(30),
INTPTLAT varchar(30),
INTPTLONG varchar(30)
);

\COPY acs_place FROM '/bulk/2017_Gaz_place_national.txt';
--COPY 29578

* TODO:
** Find acs place 2016 data for population
** Find larger acs files, ideally at the place level
** Provide more documentation on POSTGIS & geocoding

2017-10-16:
* Exported maps of points from the Bay Area each year.
** Map used location: E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS\Jeemin_Bay_Area Points_Every_Year\BayAreaEveryYearMap
** Zoom scale: 1:650.000
* Location of Bay Area Points png files:
E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS\Jeemin_Bay_Area Points_Every_Year

2017-10-10:
*Discoveries/ Struggles regarding.gdb to .shp file conversion:
** Esri Production Mapping (costly)
*** License needs to be purchasesd: http://www.esri.com/software/arcgis/extensions/production-mapping/pricing
** Use ogr2ogr from gdal package
*** https://gis.stackexchange.com/questions/14432/migrating-geodatabase-data-into-postgis-without-esri-apps
*** Command: ogr2ogr -f "ESRI Shapefile" [Destination of shapefile] [path to gdb file]
*** Problem installing gdal

2017-10-09:
* TODO'S:
** Downloading data onto tigertest
*** Road
*** Railway
*** Coastline
*** Instructions: http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation#Bulk_Download_TIGER_Shapefiles
** Configure census data from American Community Survey (ACS)
*** 1) Work out what data is of our interest (confirm ACS)
*** 2) Determine appropriate shape file unit:
**** census block vs. census block group vs. census track
*** 3) Load into tigertest

* Done:
** Downloaded data from https://www.census.gov/cgi-bin/geo/shapefiles/index.php
tl_2017_us_coastline -- 4209
tl_2017_us_primaryroads -- 11574
tl_2017_us_rails -- 176237
** Link found to potentially download ACS data: https://www.census.gov/geo/maps-data/data/tiger-data.html
*** But most files on it come with .gdb extension and not .shp

2017-10-03:
* Installed PostGIS & is now visible on pgAdmin III

* ArcGIS (connect to postgis database):
** 1) Open ArcMap
** 2) Either open blank or open existing file/project
** 3) Click on 'Add Data' button with a cross and a yellow diamond (under Selection toolbar)
** 4) Go to the top-most directory by pressing on the arrow that points left-then-up (on the left of home button)
** 5) Click on 'Database Connections'
** 6) Click on 'Add Database Connection' (if Connection to localhost.sde) does not exist already)
** 7) Fill in the following fields:
*** Database Platform: PostgreSQL
*** Instance: localhost
*** User name: postgres
*** Password:
*** Database: tigertest
** 8) Press 'OK'
** 9) Now you'll have 'Connection to localhost.sde' in your Database Connections
** 10) Double click on 'Connection to localhost.sde'
** 11) Double click on the table of interest
** 12) Click 'Finish'
** 13) You'll see information populated on map, as one of the 'Layers'
*** Tested with: tigertest.public.copointplacescontains

* On running & altering Oliver's script:
** Location: E:\McNair\Projects\OliverLovesCircles\src\python\vc_circles.py
** Ed manipulated file names so that underscores would replace dots (St.Louis --> St_Louis)
** Takes in instances and sweep times as part of the argument, but not impactful as those variables are hardcoded in the script
** Ran vc_circles.py with the following variables with changed values:
*** SWEEP_CYCLE_SECONDS = 10 (used to be 30)
*** NUMBER_INSTANCES = 16 (used to be 8)
** New output to be found in: E:\McNair\Projects\OliverLovesCircles\out

2017-10-02:
* Talked to Harrison & Peter regarding ArcGIS
** Currently have points plotted on Houston
** Trouble interpreting geometry type, as currently reads in from text file
** Documents located in : E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS
* Attempted to install PostGIS spatial extention from PostgreSQL but getting 'spatial database creation failed' error message.
** Referenced instructions:
*** https://www.gpsfiledepot.com/tutorials/installing-and-setting-up-postgresql-with-postgis/
*** http://www.bostongis.com/PrinterFriendly.aspx?content_name=postgis_tut01

2017-09-26:
* Created a table that maps a state to the database name.
** http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation#Translating_Table_names_to_corresponding_States
* Added more GIS-information (functions, realm & outliers to consider)
** http://mcnair.bakerinstitute.org/wiki/Urban_Start-up_Agglomeration#GIS_Resources
* Visualization in PostGIS or connecting to ArcGIS for visualization (import/export data)
* Spatial indexing:
** http://revenant.ca/www/postgis/workshop/indexing.html

2017-09-25:
* Talked to Ed about GIS, Census data, and going about determining the correctness of reported 'place.' Currently script makes a cross product of each reported place and an existing place, outputting a column of boolean value to indicate whether the reported place's coordinates fell within a place's geometric boundaries. One other way of going about this which we discussed is to first check if the reported place does fall within that place's boundaries. If it isn't, we'll go about the cross product method.

* To add documentation :
** http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation
** http://mcnair.bakerinstitute.org/wiki/Urban_Start-up_Agglomeration

* Discussed the need to maintain venture capital database.

*Relevant File paths:
**E:\McNair\Projects\Agglomeration\TestGIS.sql
**Z:\VentureCapitalData\SDCVCData\vcdb2\ProecssingCoLevelSimple.sql
**Z:\VentureCapitalData\SDCVCData\vcdb2\CitiesWithGT10Active.txt

2017-09-21:
* Functions for Linear Referencing:
'''ST_LineInterpolatePoint(geometry A, double measure)''': Returns a point interpolated along a line.
'''ST_LineLocatePoint(geometry A, geometry B)''': Returns a float between 0 and 1 representing the location of the closest point on LineString to the given Point.
'''ST_Line_Substring(geometry A, double from, double to)''': Return a linestring being a substring of the input one starting and ending at the given fractions of total 2d length.
'''ST_Locate_Along_Measure(geometry A, double measure)''': Return a derived geometry collection value with elements that match the specified measure.
'''ST_Locate_Between_Measures(geometry A, double from, double to)''': Return a derived geometry collection value with elements that match the specified range of measures inclusively.
'''ST_AddMeasure(geometry A, double from, double to)''': Return a derived geometry with measure elements linearly interpolated between the start and end points. If the geometry has no measure dimension, one is added.

*3-D Functions:
'''ST_3DClosestPoint''' — Returns the 3-dimensional point on g1 that is closest to g2. This is the first point of the 3D shortest line.
'''ST_3DDistance''' — For geometry type Returns the 3-dimensional cartesian minimum distance (based on spatial ref) between two geometries in projected units.
'''ST_3DDWithin''' — For 3d (z) geometry type Returns true if two geometries 3d distance is within number of units.
'''ST_3DDFullyWithin''' — Returns true if all of the 3D geometries are within the specified distance of one another.
'''ST_3DIntersects''' — Returns TRUE if the Geometries “spatially intersect” in 3d - only for points and linestrings
'''ST_3DLongestLine''' — Returns the 3-dimensional longest line between two geometries
'''ST_3DMaxDistance''' — For geometry type Returns the 3-dimensional cartesian maximum distance (based on spatial ref) between two geometries in projected units.
'''ST_3DShortestLine''' — Returns the 3-dimensional shortest line between two geometries

*Relevant PostgreSQL Commands:
'''\dt *.*''' Show all tables
'''\q''' Exit table

*Specifities/ Outliers to consider:
New York (decompose)
Princeton area (keep Princeton unique)
Reston, Virginia (keep)
San Diego (include La Jolla)
Silicon Valley (all distinct)

* Continue reading from: https://postgis.net/docs/postgis_installation.html

2017-09-20:
* Attended first intro to GIS course yesterday
* Updated above notes on GIS

2017-09-19:
* Useful functions for spatial joins:

'''sum(expression)''': aggregate to return a sum for a set of records
'''count(expression)''': aggregate to return the size of a set of records
'''ST_Area(geometry)''' returns the area of the polygons
'''ST_AsText(geometry)''' returns WKT text
'''ST_Buffer(geometry, distance)''': For geometry: Returns a geometry that represents all points whose distance from this Geometry is less than or equal to distance. Calculations are in the Spatial Reference System of this Geometry. For geography: Uses a planar transform wrapper.
'''ST_Contains(geometry A, geometry B)''' returns the true if geometry A contains geometry B
'''ST_Distance(geometry A, geometry B)''' returns the minimum distance between geometry A and geometry B
'''ST_DWithin(geometry A, geometry B, radius)''' returns the true if geometry A is radius distance or less from geometry B
'''ST_GeomFromText(text)''' returns geometry
'''ST_Intersection(geometry A, geometry B)''': Returns a geometry that represents the shared portion of geomA and geomB. The geography implementation does a transform to geometry to do the intersection and then transform back to WGS84
'''ST_Intersects(geometry A, geometry B)''' returns the true if geometry A intersects geometry B
'''ST_Length(linestring)''' returns the length of the linestring
'''ST_Touches(geometry A, geometry B)''' returns the true if the boundary of geometry A touches geometry B
'''ST_Within(geometry A, geometry B)''' returns the true if geometry A is within geometry B
geometry_a '''&&''' geometry_b: Returns TRUE if A’s bounding box overlaps B’s.
geometry_a '''=''' geometry_b: Returns TRUE if A’s bounding box is the same as B’s.
'''ST_SetSRID(geometry, srid)''': Sets the SRID on a geometry to a particular integer value.
'''ST_SRID(geometry)''': Returns the spatial reference identifier for the ST_Geometry as defined in spatial_ref_sys table.
'''ST_Transform(geometry, srid)''': Returns a new geometry with its coordinates transformed to the SRID referenced by the integer parameter.
'''ST_Union()''': Returns a geometry that represents the point set union of the Geometries.
'''substring(string [from int] [for int])''': PostgreSQL string function to extract substring matching SQL regular expression.
'''ST_Relate(geometry A, geometry B)''': Returns a text string representing the DE9IM relationship between the geometries.
'''ST_GeoHash(geometry A)''': Returns a text string representing the GeoHash of the bounds of the object.

*Native functions for geogrphy:
'''ST_AsText(geography)''' returns text
'''ST_GeographyFromText(text)''' returns geography
'''ST_AsBinary(geography)''' returns bytea
'''ST_GeogFromWKB(bytea)''' returns geography
'''ST_AsSVG(geography)''' returns text
'''ST_AsGML(geography)''' returns text
'''ST_AsKML(geography)''' returns text
'''ST_AsGeoJson(geography)''' returns text
'''ST_Distance(geography, geography)''' returns double
'''ST_DWithin(geography, geography, float8)''' returns boolean
'''ST_Area(geography)''' returns double
'''ST_Length(geography)''' returns double
'''ST_Covers(geography, geography)''' returns boolean
'''ST_CoveredBy(geography, geography)''' returns boolean
'''ST_Intersects(geography, geography)''' returns boolean
'''ST_Buffer(geography, float8)''' returns geography [1]
'''ST_Intersection(geography, geography)''' returns geography [1]
* Continue reading from: http://workshops.boundlessgeo.com/postgis-intro/geography.html

2017-09-18:
* Read documentation on PostGIS and tiger geocoder
* Continue reading from: http://workshops.boundlessgeo.com/postgis-intro/joins.html

2017-09-12:
* Clarified University Matching output file.
* Helped Christy with pdf-reader, capturing keywords in readable format.

2017-09-11:
* Ensured that documentation exists for the projects worked on last semester.

</onlyinclude>

===Spring 2017===

2017-04-17:
* To pull accelerators: Wrote simple python regex-based script that ran on organizations data.
Code: E:\McNair\Projects\Accelerators\Crunchbase Snapshot\accelerator keywords.py
Matched output (885 mathces) : E:\McNair\Projects\Accelerators\Crunchbase Snapshot\Jeemin_885_accel_matches

2017-04-14:
* Loaded Federal Grants Data into database

2017-04-12:
* Finishing up cleaning the columns for Federal Grant Data - NIH. The output excel files can be accessed at:
E:\McNair\Projects\Federal Grant Data\NIH\Grants
Titled:
Jeemin_combined_files 1986-2001.csv
Jeemin_combined_files 2002-2012.csv
Jeemin_combined_files 2013-2015.csv

* psql table formula:

CREATE TABLE all_grants (
APPLICATION_ID integer,
ACTIVITY varchar(3),
ADMINISTERING_IC varchar(2),
APPLICATION_TYPE varchar(1),
ARRA_FUNDED varchar(1),
AWARD_NOTICE_DATE date,
BUDGET_START date,
BUDGET_END date,
CFDA_CODE varchar(3),
CORE_PROJECT_NUM varchar(11),
ED_INST_TYPE varchar(30),
FOA_NUMBER varchar(13),
FULL_PROJECT_NUM varchar(35),
FUNDING_ICs varchar(40),
FUNDING_MECHANISM varchar(23),
FY smallint,
IC_NAME varchar(77),
NIH_SPENDING_CATS varchar(295),
ORG_CITY varchar(20),
ORG_COUNTRY varchar(16),
ORG_DEPT varchar(30),
ORG_DISTRICT smallint,
ORG_DUNS integer,
ORG_FIPS varchar(2),
ORG_NAME varchar(60),
ORG_STATE varchar(2),
ORG_ZIPCODE integer,
PHR varchar(200),
PI_IDS varchar(30),
PI_NAMEs varchar(200),
PROGRAM_OFFICER_NAME varchar(36),
PROJECT_START date,
PROJECT_END date,
PROJECT_TERMS varchar(200),
PROJECT_TITLE varchar(244),
SERIAL_NUMBER smallint,
STUDY_SECTION varchar(4),
STUDY_SECTION_NAME varchar(100),
SUBPROJECT_ID smallint,
SUFFIX varchar(2),
SUPPORT_YEAR smallint,
DIRECT_COST_AMT integer,
INDIRECT_COST_AMT integer,
TOTAL_COST integer,
TOTAL_COST_SUB_PROJECT integer
);
\COPY all_grants FROM 'Jeemin_combined_files 1986-2001.csv' WITH DELIMITER AS E'\t' HEADER NULL AS ''CSV

2017-03-29:
* Troubled by the variety of cases - separating keys by keywords will not work favorably when it hits University of California vs. University of Southern California case - find a way to match University of Southern California first (more specific ones first) - but how to generalize

2017-03-27:
* Writing code for university matches - decided to go through keys instead of each dataitem. Use keywords in each key to go through the dataitem - misspellings are currently unaccounted for.

2017-03-24:
* Discussed with Julia & Meghana about university keys to use to count # of occurrences, including aliases and misspellings
* Thoughts: to use a scoring metric with a key of UNIVERSITY OF CALIFORNIA SYSTEM, it should have a 'better' score when compared to MATHEMATICAL SCIENCES PUBLISHERS C/O UNIVERSITY OF CALIFORNIA BERKELEY or CALIFORNIA AT LOS ANGELES, UNVIERSITY OF than when compared to UNIVERSITY OF SOUTHERN CALIFORNIA, which may pose a challenge when attempting to implement this in a more general sense. In normalizing a string, strip "THE", "," and split words by spaces and compare each keyword from the two strings. Deciding on which strings to compare will be another issue - length (within some range maybe) could be an option.
* Federal Grant Data XML Parser was rerun - same output textfiles

2017-03-22:
* Read string matching & calculating distance, below are relevant links
* [http://www.cs.cmu.edu/~wcohen/postscript/ijcai-ws-2003.pdf]
* [http://web.archive.org/web/20081224234350/http://www.dcs.shef.ac.uk/~sam/stringmetrics.html]

2017-03-22:
* Talked to Julia about universal matcher, want to combine all University of California's to University of California, The Regents of
* Converted crunchbase2013 data from mySQL to PostgreSQL, but having trouble with the last table - cb_relationships, complains about syntax error at or near some places - but generally all tables exist in database called crunchbase
* Federal Grant Data XML Parser was run - the three output textfiles can be found in E:\McNair\Projects\Federal Grant Data\NSF

2017-03-16:
* Further documented [[University Patent Matching]]
* Finished writing XML Parser

2017-03-15:
* Todo: write a wikipage on possible input/output info on string matcher
* Wrote part of XML parser, extracted yearly data into E:\McNair\Projects\Federal Grant Data\NSF\NSF Extracted Data (up to year 2010)

2017-03-14:
* Started pulling academy cases but there are too many cases to worry about, in terms of institution of interest. A document is located in E:\McNair\Projects\University Patents\academies_verify_cases.txt
* Need Julia/Meghana to look through the hits and see which are relevant & extract pattern from there.
* Having trouble outputting txt file without double quotes around every line.
* Thinking that one text file should be output for all keywords instead of having one each, to avoid overlap (ex) COLLEGE and UNIVERSITY are both keywords; ALBERT EINSTEIN COLLEGE OF YESHIVA UNIVERSITY will be hit twice if it were counted as two separate instances, one accounting for COLLEGE and the other for UNIVERSITY) - either in the form of if-elseif statements or one big regex check.

2017-03-13:
* For University Patent Data Matching - matched SCHOOL (output: E:\McNair\Projects\University Patents\school_pulled_from_assignee_list_USA) and matched INSTITUTE(output: E:\McNair\Projects\University Patents\institute_pulled_from_assignee_list_USA).
* [[University Patent Matching]]
* To be worked on later: Grant XML parsing & general name matcher

2017-03-08:
* Wrote regex pattern that identifies all "university" matchings - can be found in E:\McNair\Projects\University Patents\university_pulled_from_assignee_list_USA -- is an output file
* Talked to Sonia, but didn't come to solid conclusion on identifying whether key words associate with city or country by running a python function
* Output sql tables from finished run of Jeemin_FDATrial_as_key_data_ripping.py
* Ran through assigneelist_USA.txt to see how many different ways UNIVERSITY could be spelled wrong. There were many.
* Tried to logic through creating a pattern that could catch all different versions of UNIVERSITY. Discuss further on whether UNIVERSITIES and those that include UNIVERSITIES but include INC in the end should be pulled as relevant information

2017-03-06:
* [[Installing python in a database]]
* Added building Python function section to [[Working with PostgreSQL]] at the bottom of the page.
* Ran FDA Trial data ripping again, as the text output files were wiped.
* Plan on discussing with Julia and Meghana again about pulling universities and other relevant institutions from the Assignee List USA.
* Talked to Sonia about pulling city, state, zipcode information, hence python was installed in a database. Will work with Sonia on Wednesday afternoon and see how best a regex function could be implemented

2017-03-03:
* Attempted to output sql tables

2017-03-01:
* Started re-running Jeemin_FDATrial_as_key_data_ripping.py

2017-02-27:
* Finished producing tables from Jeemin_FDATrial_as_key_data_ripping.py
* Talked to Julia about LinkedIn data extracting - to be discussed further with Julia & Peter.
* Started web crawler for Wikipedia - currently pulls Endowment, Academic staff, students, undergraduates, and postgraduates info found on Rice Wikipedia page. Can be found in : E:\McNair\Projects\University Patents\Jeemin_University_wikipedia_crawler.py

2017-02-24:
* Continued working on producing multiple tables - first two are done. Was working on location, as there are multiple location tags per location.

2017-02-22:
* Finished Jeemin_FDATrial_as_key_data_ripping.py (E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_FDATrial_as_key_data_ripping.py), which outputs to E:\McNair\Projects\FDA Trials\Jeemin_Project\general_data_ripping_output.txt; TODO: output four different tables & replace the write in the same for-loop as going through each file

2017-02-20:
* Continued working on Jeemin_FDATrial_as_key_data_ripping.py to find tags and place all of those information in a list. The other zipcode file did not finish executing after 2+ hours of running it - considering the possibility of splitting the record file into smaller bits, or running the processing on a faster machine.

2017-02-17:
* Completed code for counting the number of occurrences for each unique zipcode. (currently titled & located: E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_Running_File.py). It has been running for 20+min because of the comprehensive XML data files. Meanwhile started coding to create a dictionary with the keys corresponding to each unique trial ID, mapped to every other information (location, sponsors, phase, drugs ...etc.) (currently titled & located: E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_FDATrial_as_key_data_ripping.py).

2017-02-15:
* Discussed with Catherine what to do with FDA Trial data and decided to have a dictionary with zip-codes as keys and number of trials occurred in that zipcode as values. Was still attempting to loop through the files without the code having to exist in the same directory as the XML files. Plan to write to excel via tsv, with zip-code as one column and # of occurrence as the other.

2017-02-13:
* Goals (for trials): 1) Build ER Diagram 2) For each entity, get XML snippet 3) Build a parser/ripper for single file; the python parser can be found at: E:\McNair\Projects\FDA Trials\Jeemin_Project

2017-02-08:
* Attempted to come up with possible cases for locating the description of accelerators - pick up from extracting bodies of text from the about page (given that it exists)
* [[Trial Data Project]]

2017-02-06:
* Set up wikiPage & remote desktop.
* Started working on python version of web crawler. So far it successfully prints out a catchphrase/ description for one website. To be worked on. The python file can be found in: E:\McNair\Projects\Accelerators\Python WebCrawler\webcrawlerpython.py

[[Category:Work Log]]

Jeemin Sim (Work Log)

2018-01-11T20:09:10Z

JeeminS:

[[Jeemin Sim]] [[Work Logs]] [[Jeemin Sim (WorkLog)|(log page)]]

===Spring 2018===
2017-01-11:
*Was briefed on [[Urban Start-up Agglomeration and Venture Capital Investment]]

===Fall 2017===
<onlyinclude>

2017-12-05:
* Uploaded tables with KML files in TIF folder in bulk(E:) drive
** Allegheny County
** Atlanta
** Chicago
** Columbus, OH
** Dublin, OH
** Vermont
** Washington, D.C.

*Updated documentation for:
** http://mcnair.bakerinstitute.org/wiki/TIF_Project#Uploading_TIF_Data_onto_database_.28tigertest.29

* shp2pgsql needs to be installed to upload Shapefiles to PostgreSQL database
** applies for Houston, TX and Dallas, TX
** DONE

2017-12-04:
* To upload KML file into database with specified table name:
researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
* chicagotif table now resides in tigertest database

2017-11-30:
* Displayed map with TIF districts and startups
* Location:
E:\McNair\Projects\Agglomeration\TIF\McNair_Color_Chicago_TIF_and_startups.png
* Added information and steps to ArcMap/ArcGIS documentation page
* Create a project page for 'Working with POSTGIS' and add instructions for uploading KML file onto POSTGIS
** Command used :
*** Logged in as : researcher@McNairDBServ
/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml
* chicago TIF kml file currently downloaded in tigertest with a table name of Layer0
** Figure out how to change layer name while loading kml file
** Instructions pulled from : http://wiki.wildsong.biz/index.php/Loading_data_into_PostGIS#Loading_data_from_KMZ_files

2017-11-28:
* Created and edited [[ArcMap / ArcGIS Documentation]]
* Plotted points for TIFS and Startups in Chicago in one map.
** Location:
E:\McNair\Projects\Agglomeration\TIF\Jeemin_Chicago_TIF_and_Startups_Attempt1
* Used https://mygeodata.cloud/result to convert from KML file to CSV (which was then saved as txt file to be uploaded onto ArcMap)
* Text files located in Local Disk (C:) Drive

2017-11-27:
*Download Chicago TIF data

2017-11-13:
*

2017-11-09:
* Notes on data downloaded:
** Year 2010-2012 data are based on total population, not 25 yrs or over (case for all other tables)
*** Record appears five times total, with same exact column name
***For exmaple: 'Total; Estimate; High school graduate (includes equivalency)' appears five times, with different values.
* TODO:
** Make Projects page for ACS Data
***[[American Community Survey (ACS) Data]]

2017-11-07:
*Yesterday, narrowed down columns of interest from ACS_S1501_educationattain_2016 table.

Id
Id2
Geography
Total; Estimate; Population 25 years and over
Total; Estimate; Population 25 years and over - High school graduate (includes equivalency)
Total; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Total; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Percent; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Total; Estimate; Population 25 years and over - Associate's degree
Total; Margin of Error; Population 25 years and over - Associate's degree
Percent; Estimate; Population 25 years and over - Associate's degree
Percent; Margin of Error; Population 25 years and over - Associate's degree
Total; Estimate; Population 25 years and over - Bachelor's degree
Total; Margin of Error; Population 25 years and over - Bachelor's degree
Percent; Estimate; Population 25 years and over - Bachelor's degree
Percent; Margin of Error; Population 25 years and over - Bachelor's degree
Total; Estimate; Population 25 years and over - Graduate or professional degree
Total; Margin of Error; Population 25 years and over - Graduate or professional degree
Percent; Estimate; Population 25 years and over - Graduate or professional degree
Percent; Margin of Error; Population 25 years and over - Graduate or professional degree
Percent; Estimate; Percent high school graduate or higher
Percent; Margin of Error; Percent high school graduate or higher
Percent; Estimate; Percent bachelor's degree or higher
Percent; Margin of Error; Percent bachelor's degree or higher

*Complications:
**For csv files corresponding to years 2015 & 2016, all of the above columns exist.
**For csv files corresponding to years 2005 - 2014, no 'Percent' columns exist
*** Instead their 'Total' columns are percentage values
**For csv file corresponding to year 2005, columns regarding Graduate or professional degree are labeled differently.
**2012 data doesn't correspond to Population 25 years and over.

*Temporary Solution:
**Since the above problems may be specific to this set of tables, will go through csv files and adjust columns.

*Python script location:
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data\pullCertainColumns.py

2017-10-31:
* Finished doanloading files from ACS.
* Started loading tables into tigertest.
* Commands run could be found in
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data\DataLoading_SQL_Commands.txt

2017-10-30:
* Downloaded data from ACS, to be continued
* File path:
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data
* Fields of interest:
S1401 SCHOOL ENROLLMENT
S1501 EDUCATIONAL ATTAINMENT
S2301 EMPLOYMENT STATUS
B01003 TOTAL POPULATION
B02001 RACE
B07201 GEOGRAPHICAL MOBILITY
B08303 TRAVEL TIME TO WORK
B19013 MEDIAN HOUSEHOLD INCOME
B19053 SELF-EMPLOYMENT INCOME IN THE PAST 12 MONTHS FOR HOUSEHOLDS
B19083 GINI INDEX OF INCOME INEQUALITY
B25003 TENURE
B25105 MEDIAN MONTHLY HOUSING COSTS
B28011 INTERNET SUBSCRIPTIONS IN HOUSEHOLD
G001 GEOGRAPHIC IDENTIFIERS

2017-10-23:
* Talked to Ed with Peter & Oliver about upcoming tasks & projects.
* Loaded acs_place table 2017 (does not contain population) on tigertest.
** SQL commands used:

DROP TABLE acs_place;

CREATE TABLE acs_place (
USPS varchar(5),
GEOID varchar(30),
ANSICODE varchar(30),
NAME varchar(100),
LSAD varchar(30),
FUNCSTAT varchar(10),
ALAND varchar(30),
AWATER varchar(30),
ALAND_SQMI varchar(30),
AWATER_SQMI varchar(30),
INTPTLAT varchar(30),
INTPTLONG varchar(30)
);

\COPY acs_place FROM '/bulk/2017_Gaz_place_national.txt';
--COPY 29578

* TODO:
** Find acs place 2016 data for population
** Find larger acs files, ideally at the place level
** Provide more documentation on POSTGIS & geocoding

2017-10-16:
* Exported maps of points from the Bay Area each year.
** Map used location: E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS\Jeemin_Bay_Area Points_Every_Year\BayAreaEveryYearMap
** Zoom scale: 1:650.000
* Location of Bay Area Points png files:
E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS\Jeemin_Bay_Area Points_Every_Year

2017-10-10:
*Discoveries/ Struggles regarding.gdb to .shp file conversion:
** Esri Production Mapping (costly)
*** License needs to be purchasesd: http://www.esri.com/software/arcgis/extensions/production-mapping/pricing
** Use ogr2ogr from gdal package
*** https://gis.stackexchange.com/questions/14432/migrating-geodatabase-data-into-postgis-without-esri-apps
*** Command: ogr2ogr -f "ESRI Shapefile" [Destination of shapefile] [path to gdb file]
*** Problem installing gdal

2017-10-09:
* TODO'S:
** Downloading data onto tigertest
*** Road
*** Railway
*** Coastline
*** Instructions: http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation#Bulk_Download_TIGER_Shapefiles
** Configure census data from American Community Survey (ACS)
*** 1) Work out what data is of our interest (confirm ACS)
*** 2) Determine appropriate shape file unit:
**** census block vs. census block group vs. census track
*** 3) Load into tigertest

* Done:
** Downloaded data from https://www.census.gov/cgi-bin/geo/shapefiles/index.php
tl_2017_us_coastline -- 4209
tl_2017_us_primaryroads -- 11574
tl_2017_us_rails -- 176237
** Link found to potentially download ACS data: https://www.census.gov/geo/maps-data/data/tiger-data.html
*** But most files on it come with .gdb extension and not .shp

2017-10-03:
* Installed PostGIS & is now visible on pgAdmin III

* ArcGIS (connect to postgis database):
** 1) Open ArcMap
** 2) Either open blank or open existing file/project
** 3) Click on 'Add Data' button with a cross and a yellow diamond (under Selection toolbar)
** 4) Go to the top-most directory by pressing on the arrow that points left-then-up (on the left of home button)
** 5) Click on 'Database Connections'
** 6) Click on 'Add Database Connection' (if Connection to localhost.sde) does not exist already)
** 7) Fill in the following fields:
*** Database Platform: PostgreSQL
*** Instance: localhost
*** User name: postgres
*** Password:
*** Database: tigertest
** 8) Press 'OK'
** 9) Now you'll have 'Connection to localhost.sde' in your Database Connections
** 10) Double click on 'Connection to localhost.sde'
** 11) Double click on the table of interest
** 12) Click 'Finish'
** 13) You'll see information populated on map, as one of the 'Layers'
*** Tested with: tigertest.public.copointplacescontains

* On running & altering Oliver's script:
** Location: E:\McNair\Projects\OliverLovesCircles\src\python\vc_circles.py
** Ed manipulated file names so that underscores would replace dots (St.Louis --> St_Louis)
** Takes in instances and sweep times as part of the argument, but not impactful as those variables are hardcoded in the script
** Ran vc_circles.py with the following variables with changed values:
*** SWEEP_CYCLE_SECONDS = 10 (used to be 30)
*** NUMBER_INSTANCES = 16 (used to be 8)
** New output to be found in: E:\McNair\Projects\OliverLovesCircles\out

2017-10-02:
* Talked to Harrison & Peter regarding ArcGIS
** Currently have points plotted on Houston
** Trouble interpreting geometry type, as currently reads in from text file
** Documents located in : E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS
* Attempted to install PostGIS spatial extention from PostgreSQL but getting 'spatial database creation failed' error message.
** Referenced instructions:
*** https://www.gpsfiledepot.com/tutorials/installing-and-setting-up-postgresql-with-postgis/
*** http://www.bostongis.com/PrinterFriendly.aspx?content_name=postgis_tut01

2017-09-26:
* Created a table that maps a state to the database name.
** http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation#Translating_Table_names_to_corresponding_States
* Added more GIS-information (functions, realm & outliers to consider)
** http://mcnair.bakerinstitute.org/wiki/Urban_Start-up_Agglomeration#GIS_Resources
* Visualization in PostGIS or connecting to ArcGIS for visualization (import/export data)
* Spatial indexing:
** http://revenant.ca/www/postgis/workshop/indexing.html

2017-09-25:
* Talked to Ed about GIS, Census data, and going about determining the correctness of reported 'place.' Currently script makes a cross product of each reported place and an existing place, outputting a column of boolean value to indicate whether the reported place's coordinates fell within a place's geometric boundaries. One other way of going about this which we discussed is to first check if the reported place does fall within that place's boundaries. If it isn't, we'll go about the cross product method.

* To add documentation :
** http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation
** http://mcnair.bakerinstitute.org/wiki/Urban_Start-up_Agglomeration

* Discussed the need to maintain venture capital database.

*Relevant File paths:
**E:\McNair\Projects\Agglomeration\TestGIS.sql
**Z:\VentureCapitalData\SDCVCData\vcdb2\ProecssingCoLevelSimple.sql
**Z:\VentureCapitalData\SDCVCData\vcdb2\CitiesWithGT10Active.txt

2017-09-21:
* Functions for Linear Referencing:
'''ST_LineInterpolatePoint(geometry A, double measure)''': Returns a point interpolated along a line.
'''ST_LineLocatePoint(geometry A, geometry B)''': Returns a float between 0 and 1 representing the location of the closest point on LineString to the given Point.
'''ST_Line_Substring(geometry A, double from, double to)''': Return a linestring being a substring of the input one starting and ending at the given fractions of total 2d length.
'''ST_Locate_Along_Measure(geometry A, double measure)''': Return a derived geometry collection value with elements that match the specified measure.
'''ST_Locate_Between_Measures(geometry A, double from, double to)''': Return a derived geometry collection value with elements that match the specified range of measures inclusively.
'''ST_AddMeasure(geometry A, double from, double to)''': Return a derived geometry with measure elements linearly interpolated between the start and end points. If the geometry has no measure dimension, one is added.

*3-D Functions:
'''ST_3DClosestPoint''' — Returns the 3-dimensional point on g1 that is closest to g2. This is the first point of the 3D shortest line.
'''ST_3DDistance''' — For geometry type Returns the 3-dimensional cartesian minimum distance (based on spatial ref) between two geometries in projected units.
'''ST_3DDWithin''' — For 3d (z) geometry type Returns true if two geometries 3d distance is within number of units.
'''ST_3DDFullyWithin''' — Returns true if all of the 3D geometries are within the specified distance of one another.
'''ST_3DIntersects''' — Returns TRUE if the Geometries “spatially intersect” in 3d - only for points and linestrings
'''ST_3DLongestLine''' — Returns the 3-dimensional longest line between two geometries
'''ST_3DMaxDistance''' — For geometry type Returns the 3-dimensional cartesian maximum distance (based on spatial ref) between two geometries in projected units.
'''ST_3DShortestLine''' — Returns the 3-dimensional shortest line between two geometries

*Relevant PostgreSQL Commands:
'''\dt *.*''' Show all tables
'''\q''' Exit table

*Specifities/ Outliers to consider:
New York (decompose)
Princeton area (keep Princeton unique)
Reston, Virginia (keep)
San Diego (include La Jolla)
Silicon Valley (all distinct)

* Continue reading from: https://postgis.net/docs/postgis_installation.html

2017-09-20:
* Attended first intro to GIS course yesterday
* Updated above notes on GIS

2017-09-19:
* Useful functions for spatial joins:

'''sum(expression)''': aggregate to return a sum for a set of records
'''count(expression)''': aggregate to return the size of a set of records
'''ST_Area(geometry)''' returns the area of the polygons
'''ST_AsText(geometry)''' returns WKT text
'''ST_Buffer(geometry, distance)''': For geometry: Returns a geometry that represents all points whose distance from this Geometry is less than or equal to distance. Calculations are in the Spatial Reference System of this Geometry. For geography: Uses a planar transform wrapper.
'''ST_Contains(geometry A, geometry B)''' returns the true if geometry A contains geometry B
'''ST_Distance(geometry A, geometry B)''' returns the minimum distance between geometry A and geometry B
'''ST_DWithin(geometry A, geometry B, radius)''' returns the true if geometry A is radius distance or less from geometry B
'''ST_GeomFromText(text)''' returns geometry
'''ST_Intersection(geometry A, geometry B)''': Returns a geometry that represents the shared portion of geomA and geomB. The geography implementation does a transform to geometry to do the intersection and then transform back to WGS84
'''ST_Intersects(geometry A, geometry B)''' returns the true if geometry A intersects geometry B
'''ST_Length(linestring)''' returns the length of the linestring
'''ST_Touches(geometry A, geometry B)''' returns the true if the boundary of geometry A touches geometry B
'''ST_Within(geometry A, geometry B)''' returns the true if geometry A is within geometry B
geometry_a '''&&''' geometry_b: Returns TRUE if A’s bounding box overlaps B’s.
geometry_a '''=''' geometry_b: Returns TRUE if A’s bounding box is the same as B’s.
'''ST_SetSRID(geometry, srid)''': Sets the SRID on a geometry to a particular integer value.
'''ST_SRID(geometry)''': Returns the spatial reference identifier for the ST_Geometry as defined in spatial_ref_sys table.
'''ST_Transform(geometry, srid)''': Returns a new geometry with its coordinates transformed to the SRID referenced by the integer parameter.
'''ST_Union()''': Returns a geometry that represents the point set union of the Geometries.
'''substring(string [from int] [for int])''': PostgreSQL string function to extract substring matching SQL regular expression.
'''ST_Relate(geometry A, geometry B)''': Returns a text string representing the DE9IM relationship between the geometries.
'''ST_GeoHash(geometry A)''': Returns a text string representing the GeoHash of the bounds of the object.

*Native functions for geogrphy:
'''ST_AsText(geography)''' returns text
'''ST_GeographyFromText(text)''' returns geography
'''ST_AsBinary(geography)''' returns bytea
'''ST_GeogFromWKB(bytea)''' returns geography
'''ST_AsSVG(geography)''' returns text
'''ST_AsGML(geography)''' returns text
'''ST_AsKML(geography)''' returns text
'''ST_AsGeoJson(geography)''' returns text
'''ST_Distance(geography, geography)''' returns double
'''ST_DWithin(geography, geography, float8)''' returns boolean
'''ST_Area(geography)''' returns double
'''ST_Length(geography)''' returns double
'''ST_Covers(geography, geography)''' returns boolean
'''ST_CoveredBy(geography, geography)''' returns boolean
'''ST_Intersects(geography, geography)''' returns boolean
'''ST_Buffer(geography, float8)''' returns geography [1]
'''ST_Intersection(geography, geography)''' returns geography [1]
* Continue reading from: http://workshops.boundlessgeo.com/postgis-intro/geography.html

2017-09-18:
* Read documentation on PostGIS and tiger geocoder
* Continue reading from: http://workshops.boundlessgeo.com/postgis-intro/joins.html

2017-09-12:
* Clarified University Matching output file.
* Helped Christy with pdf-reader, capturing keywords in readable format.

2017-09-11:
* Ensured that documentation exists for the projects worked on last semester.

</onlyinclude>

===Spring 2017===

2017-04-17:
* To pull accelerators: Wrote simple python regex-based script that ran on organizations data.
Code: E:\McNair\Projects\Accelerators\Crunchbase Snapshot\accelerator keywords.py
Matched output (885 mathces) : E:\McNair\Projects\Accelerators\Crunchbase Snapshot\Jeemin_885_accel_matches

2017-04-14:
* Loaded Federal Grants Data into database

2017-04-12:
* Finishing up cleaning the columns for Federal Grant Data - NIH. The output excel files can be accessed at:
E:\McNair\Projects\Federal Grant Data\NIH\Grants
Titled:
Jeemin_combined_files 1986-2001.csv
Jeemin_combined_files 2002-2012.csv
Jeemin_combined_files 2013-2015.csv

* psql table formula:

CREATE TABLE all_grants (
APPLICATION_ID integer,
ACTIVITY varchar(3),
ADMINISTERING_IC varchar(2),
APPLICATION_TYPE varchar(1),
ARRA_FUNDED varchar(1),
AWARD_NOTICE_DATE date,
BUDGET_START date,
BUDGET_END date,
CFDA_CODE varchar(3),
CORE_PROJECT_NUM varchar(11),
ED_INST_TYPE varchar(30),
FOA_NUMBER varchar(13),
FULL_PROJECT_NUM varchar(35),
FUNDING_ICs varchar(40),
FUNDING_MECHANISM varchar(23),
FY smallint,
IC_NAME varchar(77),
NIH_SPENDING_CATS varchar(295),
ORG_CITY varchar(20),
ORG_COUNTRY varchar(16),
ORG_DEPT varchar(30),
ORG_DISTRICT smallint,
ORG_DUNS integer,
ORG_FIPS varchar(2),
ORG_NAME varchar(60),
ORG_STATE varchar(2),
ORG_ZIPCODE integer,
PHR varchar(200),
PI_IDS varchar(30),
PI_NAMEs varchar(200),
PROGRAM_OFFICER_NAME varchar(36),
PROJECT_START date,
PROJECT_END date,
PROJECT_TERMS varchar(200),
PROJECT_TITLE varchar(244),
SERIAL_NUMBER smallint,
STUDY_SECTION varchar(4),
STUDY_SECTION_NAME varchar(100),
SUBPROJECT_ID smallint,
SUFFIX varchar(2),
SUPPORT_YEAR smallint,
DIRECT_COST_AMT integer,
INDIRECT_COST_AMT integer,
TOTAL_COST integer,
TOTAL_COST_SUB_PROJECT integer
);
\COPY all_grants FROM 'Jeemin_combined_files 1986-2001.csv' WITH DELIMITER AS E'\t' HEADER NULL AS ''CSV

2017-03-29:
* Troubled by the variety of cases - separating keys by keywords will not work favorably when it hits University of California vs. University of Southern California case - find a way to match University of Southern California first (more specific ones first) - but how to generalize

2017-03-27:
* Writing code for university matches - decided to go through keys instead of each dataitem. Use keywords in each key to go through the dataitem - misspellings are currently unaccounted for.

2017-03-24:
* Discussed with Julia & Meghana about university keys to use to count # of occurrences, including aliases and misspellings
* Thoughts: to use a scoring metric with a key of UNIVERSITY OF CALIFORNIA SYSTEM, it should have a 'better' score when compared to MATHEMATICAL SCIENCES PUBLISHERS C/O UNIVERSITY OF CALIFORNIA BERKELEY or CALIFORNIA AT LOS ANGELES, UNVIERSITY OF than when compared to UNIVERSITY OF SOUTHERN CALIFORNIA, which may pose a challenge when attempting to implement this in a more general sense. In normalizing a string, strip "THE", "," and split words by spaces and compare each keyword from the two strings. Deciding on which strings to compare will be another issue - length (within some range maybe) could be an option.
* Federal Grant Data XML Parser was rerun - same output textfiles

2017-03-22:
* Read string matching & calculating distance, below are relevant links
* [http://www.cs.cmu.edu/~wcohen/postscript/ijcai-ws-2003.pdf]
* [http://web.archive.org/web/20081224234350/http://www.dcs.shef.ac.uk/~sam/stringmetrics.html]

2017-03-22:
* Talked to Julia about universal matcher, want to combine all University of California's to University of California, The Regents of
* Converted crunchbase2013 data from mySQL to PostgreSQL, but having trouble with the last table - cb_relationships, complains about syntax error at or near some places - but generally all tables exist in database called crunchbase
* Federal Grant Data XML Parser was run - the three output textfiles can be found in E:\McNair\Projects\Federal Grant Data\NSF

2017-03-16:
* Further documented [[University Patent Matching]]
* Finished writing XML Parser

2017-03-15:
* Todo: write a wikipage on possible input/output info on string matcher
* Wrote part of XML parser, extracted yearly data into E:\McNair\Projects\Federal Grant Data\NSF\NSF Extracted Data (up to year 2010)

2017-03-14:
* Started pulling academy cases but there are too many cases to worry about, in terms of institution of interest. A document is located in E:\McNair\Projects\University Patents\academies_verify_cases.txt
* Need Julia/Meghana to look through the hits and see which are relevant & extract pattern from there.
* Having trouble outputting txt file without double quotes around every line.
* Thinking that one text file should be output for all keywords instead of having one each, to avoid overlap (ex) COLLEGE and UNIVERSITY are both keywords; ALBERT EINSTEIN COLLEGE OF YESHIVA UNIVERSITY will be hit twice if it were counted as two separate instances, one accounting for COLLEGE and the other for UNIVERSITY) - either in the form of if-elseif statements or one big regex check.

2017-03-13:
* For University Patent Data Matching - matched SCHOOL (output: E:\McNair\Projects\University Patents\school_pulled_from_assignee_list_USA) and matched INSTITUTE(output: E:\McNair\Projects\University Patents\institute_pulled_from_assignee_list_USA).
* [[University Patent Matching]]
* To be worked on later: Grant XML parsing & general name matcher

2017-03-08:
* Wrote regex pattern that identifies all "university" matchings - can be found in E:\McNair\Projects\University Patents\university_pulled_from_assignee_list_USA -- is an output file
* Talked to Sonia, but didn't come to solid conclusion on identifying whether key words associate with city or country by running a python function
* Output sql tables from finished run of Jeemin_FDATrial_as_key_data_ripping.py
* Ran through assigneelist_USA.txt to see how many different ways UNIVERSITY could be spelled wrong. There were many.
* Tried to logic through creating a pattern that could catch all different versions of UNIVERSITY. Discuss further on whether UNIVERSITIES and those that include UNIVERSITIES but include INC in the end should be pulled as relevant information

2017-03-06:
* [[Installing python in a database]]
* Added building Python function section to [[Working with PostgreSQL]] at the bottom of the page.
* Ran FDA Trial data ripping again, as the text output files were wiped.
* Plan on discussing with Julia and Meghana again about pulling universities and other relevant institutions from the Assignee List USA.
* Talked to Sonia about pulling city, state, zipcode information, hence python was installed in a database. Will work with Sonia on Wednesday afternoon and see how best a regex function could be implemented

2017-03-03:
* Attempted to output sql tables

2017-03-01:
* Started re-running Jeemin_FDATrial_as_key_data_ripping.py

2017-02-27:
* Finished producing tables from Jeemin_FDATrial_as_key_data_ripping.py
* Talked to Julia about LinkedIn data extracting - to be discussed further with Julia & Peter.
* Started web crawler for Wikipedia - currently pulls Endowment, Academic staff, students, undergraduates, and postgraduates info found on Rice Wikipedia page. Can be found in : E:\McNair\Projects\University Patents\Jeemin_University_wikipedia_crawler.py

2017-02-24:
* Continued working on producing multiple tables - first two are done. Was working on location, as there are multiple location tags per location.

2017-02-22:
* Finished Jeemin_FDATrial_as_key_data_ripping.py (E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_FDATrial_as_key_data_ripping.py), which outputs to E:\McNair\Projects\FDA Trials\Jeemin_Project\general_data_ripping_output.txt; TODO: output four different tables & replace the write in the same for-loop as going through each file

2017-02-20:
* Continued working on Jeemin_FDATrial_as_key_data_ripping.py to find tags and place all of those information in a list. The other zipcode file did not finish executing after 2+ hours of running it - considering the possibility of splitting the record file into smaller bits, or running the processing on a faster machine.

2017-02-17:
* Completed code for counting the number of occurrences for each unique zipcode. (currently titled & located: E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_Running_File.py). It has been running for 20+min because of the comprehensive XML data files. Meanwhile started coding to create a dictionary with the keys corresponding to each unique trial ID, mapped to every other information (location, sponsors, phase, drugs ...etc.) (currently titled & located: E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_FDATrial_as_key_data_ripping.py).

2017-02-15:
* Discussed with Catherine what to do with FDA Trial data and decided to have a dictionary with zip-codes as keys and number of trials occurred in that zipcode as values. Was still attempting to loop through the files without the code having to exist in the same directory as the XML files. Plan to write to excel via tsv, with zip-code as one column and # of occurrence as the other.

2017-02-13:
* Goals (for trials): 1) Build ER Diagram 2) For each entity, get XML snippet 3) Build a parser/ripper for single file; the python parser can be found at: E:\McNair\Projects\FDA Trials\Jeemin_Project

2017-02-08:
* Attempted to come up with possible cases for locating the description of accelerators - pick up from extracting bodies of text from the about page (given that it exists)
* [[Trial Data Project]]

2017-02-06:
* Set up wikiPage & remote desktop.
* Started working on python version of web crawler. So far it successfully prints out a catchphrase/ description for one website. To be worked on. The python file can be found in: E:\McNair\Projects\Accelerators\Python WebCrawler\webcrawlerpython.py

[[Category:Work Log]]

TIF Project

2017-12-07T17:12:17Z

JeeminS: /* Dallas, TX */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== Command to upload SHP file into a table in database (tigertest) ==
researcher@McNairDBServ:/bulk/Jeemin/Dallas$ shp2pgsql -I "TIFDistrict.shp" dallastif | psql -U postgres -W -d tigertest
* Run this command in a folder with all files for the table, such as files of type .shp, .dbf, .shx, .cpg, and .prj
* "TIFDistrict.shp" is the name of the shapefile and dallstif is the name of the table I wish to name
* Will be prompted for a password after running inputting this command

== To view all columns of a table ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago: Included ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

pull columns: wkb_geometry, approval_d, expiration

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta: Included ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

pull columns: wkb_geometry, start_, (say that end is 2020)

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH: Included ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

pull columns: wkb_geometry, ord_date, exp_date

=== Vermont ===

Table name: vermonttif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('vermonttif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid_1 | integer | | | | plain | |
status | character varying | | | | extended | |
tif | character varying | | | | extended | |
acresotv | double precision | | | | plain | |
parcels | integer | | | | plain | |
edtaxablev | integer | | | | plain | |
munitaxabl | integer | | | | plain | |
contactnam | character varying | | | | extended | |
contacttit | character varying | | | | extended | |
contacttel | character varying | | | | extended | |
contactema | character varying | | | | extended | |
stateconta | character varying | | | | extended | |
statetitle | character varying | | | | extended | |
statetelep | character varying | | | | extended | |
stateemail | character varying | | | | extended | |
datelastup | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Dallas, TX: Included ===

Table name: dallastif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('dallastif_gid_seq'::regclass) | plain | |
cvttxdscrp | character varying(150) | | | | extended | |
nofore | bigint | | | | plain | |
noappl | bigint | | | | plain | |
nosale | bigint | | | | plain | |
sumasvlchg | numeric | | | | main | |
sumtxvlchg | numeric | | | | main | |
sumtxodchg | numeric | | | | main | |
lastupdate | date | | | | plain | |
cvttxcd | integer | | | | plain | |
name | character varying(150) | | | | extended | |
created_us | character varying(254) | | | | extended | |
created_da | date | | | | plain | |
last_edite | character varying(254) | | | | extended | |
last_edi_1 | date | | | | plain | |
shape_star | numeric | | | | main | |
shape_stle | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

pull columns: geom, created_da, (assume 2020 as end date)

=== Houston, TX ===

Table name: houstontif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+-----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('houstontif_gid_seq'::regclass) | plain | |
perimeter | numeric | | | | main | |
name | character varying(25) | | | | extended | |
siteno | integer | | | | plain | |
cohgis_coh | numeric | | | | main | |
cohgis_c_1 | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

TIF Project

2017-12-07T17:12:03Z

JeeminS: /* Dublin, OH */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== Command to upload SHP file into a table in database (tigertest) ==
researcher@McNairDBServ:/bulk/Jeemin/Dallas$ shp2pgsql -I "TIFDistrict.shp" dallastif | psql -U postgres -W -d tigertest
* Run this command in a folder with all files for the table, such as files of type .shp, .dbf, .shx, .cpg, and .prj
* "TIFDistrict.shp" is the name of the shapefile and dallstif is the name of the table I wish to name
* Will be prompted for a password after running inputting this command

== To view all columns of a table ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago: Included ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

pull columns: wkb_geometry, approval_d, expiration

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta: Included ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

pull columns: wkb_geometry, start_, (say that end is 2020)

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH: Included ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

pull columns: wkb_geometry, ord_date, exp_date

=== Vermont ===

Table name: vermonttif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('vermonttif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid_1 | integer | | | | plain | |
status | character varying | | | | extended | |
tif | character varying | | | | extended | |
acresotv | double precision | | | | plain | |
parcels | integer | | | | plain | |
edtaxablev | integer | | | | plain | |
munitaxabl | integer | | | | plain | |
contactnam | character varying | | | | extended | |
contacttit | character varying | | | | extended | |
contacttel | character varying | | | | extended | |
contactema | character varying | | | | extended | |
stateconta | character varying | | | | extended | |
statetitle | character varying | | | | extended | |
statetelep | character varying | | | | extended | |
stateemail | character varying | | | | extended | |
datelastup | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Dallas, TX ===

Table name: dallastif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('dallastif_gid_seq'::regclass) | plain | |
cvttxdscrp | character varying(150) | | | | extended | |
nofore | bigint | | | | plain | |
noappl | bigint | | | | plain | |
nosale | bigint | | | | plain | |
sumasvlchg | numeric | | | | main | |
sumtxvlchg | numeric | | | | main | |
sumtxodchg | numeric | | | | main | |
lastupdate | date | | | | plain | |
cvttxcd | integer | | | | plain | |
name | character varying(150) | | | | extended | |
created_us | character varying(254) | | | | extended | |
created_da | date | | | | plain | |
last_edite | character varying(254) | | | | extended | |
last_edi_1 | date | | | | plain | |
shape_star | numeric | | | | main | |
shape_stle | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

pull columns: geom, created_da, (assume 2020 as end date)

=== Houston, TX ===

Table name: houstontif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+-----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('houstontif_gid_seq'::regclass) | plain | |
perimeter | numeric | | | | main | |
name | character varying(25) | | | | extended | |
siteno | integer | | | | plain | |
cohgis_coh | numeric | | | | main | |
cohgis_c_1 | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

TIF Project

2017-12-07T17:11:50Z

JeeminS: /* Atlanta */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== Command to upload SHP file into a table in database (tigertest) ==
researcher@McNairDBServ:/bulk/Jeemin/Dallas$ shp2pgsql -I "TIFDistrict.shp" dallastif | psql -U postgres -W -d tigertest
* Run this command in a folder with all files for the table, such as files of type .shp, .dbf, .shx, .cpg, and .prj
* "TIFDistrict.shp" is the name of the shapefile and dallstif is the name of the table I wish to name
* Will be prompted for a password after running inputting this command

== To view all columns of a table ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago: Included ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

pull columns: wkb_geometry, approval_d, expiration

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta: Included ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

pull columns: wkb_geometry, start_, (say that end is 2020)

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

pull columns: wkb_geometry, ord_date, exp_date

=== Vermont ===

Table name: vermonttif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('vermonttif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid_1 | integer | | | | plain | |
status | character varying | | | | extended | |
tif | character varying | | | | extended | |
acresotv | double precision | | | | plain | |
parcels | integer | | | | plain | |
edtaxablev | integer | | | | plain | |
munitaxabl | integer | | | | plain | |
contactnam | character varying | | | | extended | |
contacttit | character varying | | | | extended | |
contacttel | character varying | | | | extended | |
contactema | character varying | | | | extended | |
stateconta | character varying | | | | extended | |
statetitle | character varying | | | | extended | |
statetelep | character varying | | | | extended | |
stateemail | character varying | | | | extended | |
datelastup | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Dallas, TX ===

Table name: dallastif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('dallastif_gid_seq'::regclass) | plain | |
cvttxdscrp | character varying(150) | | | | extended | |
nofore | bigint | | | | plain | |
noappl | bigint | | | | plain | |
nosale | bigint | | | | plain | |
sumasvlchg | numeric | | | | main | |
sumtxvlchg | numeric | | | | main | |
sumtxodchg | numeric | | | | main | |
lastupdate | date | | | | plain | |
cvttxcd | integer | | | | plain | |
name | character varying(150) | | | | extended | |
created_us | character varying(254) | | | | extended | |
created_da | date | | | | plain | |
last_edite | character varying(254) | | | | extended | |
last_edi_1 | date | | | | plain | |
shape_star | numeric | | | | main | |
shape_stle | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

pull columns: geom, created_da, (assume 2020 as end date)

=== Houston, TX ===

Table name: houstontif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+-----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('houstontif_gid_seq'::regclass) | plain | |
perimeter | numeric | | | | main | |
name | character varying(25) | | | | extended | |
siteno | integer | | | | plain | |
cohgis_coh | numeric | | | | main | |
cohgis_c_1 | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

TIF Project

2017-12-07T17:11:36Z

JeeminS: /* Chicago */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== Command to upload SHP file into a table in database (tigertest) ==
researcher@McNairDBServ:/bulk/Jeemin/Dallas$ shp2pgsql -I "TIFDistrict.shp" dallastif | psql -U postgres -W -d tigertest
* Run this command in a folder with all files for the table, such as files of type .shp, .dbf, .shx, .cpg, and .prj
* "TIFDistrict.shp" is the name of the shapefile and dallstif is the name of the table I wish to name
* Will be prompted for a password after running inputting this command

== To view all columns of a table ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago: Included ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

pull columns: wkb_geometry, approval_d, expiration

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

pull columns: wkb_geometry, start_, (say that end is 2020)

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

pull columns: wkb_geometry, ord_date, exp_date

=== Vermont ===

Table name: vermonttif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('vermonttif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid_1 | integer | | | | plain | |
status | character varying | | | | extended | |
tif | character varying | | | | extended | |
acresotv | double precision | | | | plain | |
parcels | integer | | | | plain | |
edtaxablev | integer | | | | plain | |
munitaxabl | integer | | | | plain | |
contactnam | character varying | | | | extended | |
contacttit | character varying | | | | extended | |
contacttel | character varying | | | | extended | |
contactema | character varying | | | | extended | |
stateconta | character varying | | | | extended | |
statetitle | character varying | | | | extended | |
statetelep | character varying | | | | extended | |
stateemail | character varying | | | | extended | |
datelastup | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Dallas, TX ===

Table name: dallastif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('dallastif_gid_seq'::regclass) | plain | |
cvttxdscrp | character varying(150) | | | | extended | |
nofore | bigint | | | | plain | |
noappl | bigint | | | | plain | |
nosale | bigint | | | | plain | |
sumasvlchg | numeric | | | | main | |
sumtxvlchg | numeric | | | | main | |
sumtxodchg | numeric | | | | main | |
lastupdate | date | | | | plain | |
cvttxcd | integer | | | | plain | |
name | character varying(150) | | | | extended | |
created_us | character varying(254) | | | | extended | |
created_da | date | | | | plain | |
last_edite | character varying(254) | | | | extended | |
last_edi_1 | date | | | | plain | |
shape_star | numeric | | | | main | |
shape_stle | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

pull columns: geom, created_da, (assume 2020 as end date)

=== Houston, TX ===

Table name: houstontif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+-----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('houstontif_gid_seq'::regclass) | plain | |
perimeter | numeric | | | | main | |
name | character varying(25) | | | | extended | |
siteno | integer | | | | plain | |
cohgis_coh | numeric | | | | main | |
cohgis_c_1 | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

TIF Project

2017-12-07T17:11:04Z

JeeminS: /* Dallas, TX */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== Command to upload SHP file into a table in database (tigertest) ==
researcher@McNairDBServ:/bulk/Jeemin/Dallas$ shp2pgsql -I "TIFDistrict.shp" dallastif | psql -U postgres -W -d tigertest
* Run this command in a folder with all files for the table, such as files of type .shp, .dbf, .shx, .cpg, and .prj
* "TIFDistrict.shp" is the name of the shapefile and dallstif is the name of the table I wish to name
* Will be prompted for a password after running inputting this command

== To view all columns of a table ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

pull columns: wkb_geometry, approval_d, expiration

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

pull columns: wkb_geometry, start_, (say that end is 2020)

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

pull columns: wkb_geometry, ord_date, exp_date

=== Vermont ===

Table name: vermonttif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('vermonttif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid_1 | integer | | | | plain | |
status | character varying | | | | extended | |
tif | character varying | | | | extended | |
acresotv | double precision | | | | plain | |
parcels | integer | | | | plain | |
edtaxablev | integer | | | | plain | |
munitaxabl | integer | | | | plain | |
contactnam | character varying | | | | extended | |
contacttit | character varying | | | | extended | |
contacttel | character varying | | | | extended | |
contactema | character varying | | | | extended | |
stateconta | character varying | | | | extended | |
statetitle | character varying | | | | extended | |
statetelep | character varying | | | | extended | |
stateemail | character varying | | | | extended | |
datelastup | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Dallas, TX ===

Table name: dallastif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('dallastif_gid_seq'::regclass) | plain | |
cvttxdscrp | character varying(150) | | | | extended | |
nofore | bigint | | | | plain | |
noappl | bigint | | | | plain | |
nosale | bigint | | | | plain | |
sumasvlchg | numeric | | | | main | |
sumtxvlchg | numeric | | | | main | |
sumtxodchg | numeric | | | | main | |
lastupdate | date | | | | plain | |
cvttxcd | integer | | | | plain | |
name | character varying(150) | | | | extended | |
created_us | character varying(254) | | | | extended | |
created_da | date | | | | plain | |
last_edite | character varying(254) | | | | extended | |
last_edi_1 | date | | | | plain | |
shape_star | numeric | | | | main | |
shape_stle | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

pull columns: geom, created_da, (assume 2020 as end date)

=== Houston, TX ===

Table name: houstontif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+-----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('houstontif_gid_seq'::regclass) | plain | |
perimeter | numeric | | | | main | |
name | character varying(25) | | | | extended | |
siteno | integer | | | | plain | |
cohgis_coh | numeric | | | | main | |
cohgis_c_1 | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

TIF Project

2017-12-07T17:09:38Z

JeeminS: /* Dublin, OH */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== Command to upload SHP file into a table in database (tigertest) ==
researcher@McNairDBServ:/bulk/Jeemin/Dallas$ shp2pgsql -I "TIFDistrict.shp" dallastif | psql -U postgres -W -d tigertest
* Run this command in a folder with all files for the table, such as files of type .shp, .dbf, .shx, .cpg, and .prj
* "TIFDistrict.shp" is the name of the shapefile and dallstif is the name of the table I wish to name
* Will be prompted for a password after running inputting this command

== To view all columns of a table ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

pull columns: wkb_geometry, approval_d, expiration

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

pull columns: wkb_geometry, start_, (say that end is 2020)

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

pull columns: wkb_geometry, ord_date, exp_date

=== Vermont ===

Table name: vermonttif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('vermonttif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid_1 | integer | | | | plain | |
status | character varying | | | | extended | |
tif | character varying | | | | extended | |
acresotv | double precision | | | | plain | |
parcels | integer | | | | plain | |
edtaxablev | integer | | | | plain | |
munitaxabl | integer | | | | plain | |
contactnam | character varying | | | | extended | |
contacttit | character varying | | | | extended | |
contacttel | character varying | | | | extended | |
contactema | character varying | | | | extended | |
stateconta | character varying | | | | extended | |
statetitle | character varying | | | | extended | |
statetelep | character varying | | | | extended | |
stateemail | character varying | | | | extended | |
datelastup | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Dallas, TX ===

Table name: dallastif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('dallastif_gid_seq'::regclass) | plain | |
cvttxdscrp | character varying(150) | | | | extended | |
nofore | bigint | | | | plain | |
noappl | bigint | | | | plain | |
nosale | bigint | | | | plain | |
sumasvlchg | numeric | | | | main | |
sumtxvlchg | numeric | | | | main | |
sumtxodchg | numeric | | | | main | |
lastupdate | date | | | | plain | |
cvttxcd | integer | | | | plain | |
name | character varying(150) | | | | extended | |
created_us | character varying(254) | | | | extended | |
created_da | date | | | | plain | |
last_edite | character varying(254) | | | | extended | |
last_edi_1 | date | | | | plain | |
shape_star | numeric | | | | main | |
shape_stle | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

=== Houston, TX ===

Table name: houstontif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+-----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('houstontif_gid_seq'::regclass) | plain | |
perimeter | numeric | | | | main | |
name | character varying(25) | | | | extended | |
siteno | integer | | | | plain | |
cohgis_coh | numeric | | | | main | |
cohgis_c_1 | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

TIF Project

2017-12-07T16:57:42Z

JeeminS: /* Atlanta */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== Command to upload SHP file into a table in database (tigertest) ==
researcher@McNairDBServ:/bulk/Jeemin/Dallas$ shp2pgsql -I "TIFDistrict.shp" dallastif | psql -U postgres -W -d tigertest
* Run this command in a folder with all files for the table, such as files of type .shp, .dbf, .shx, .cpg, and .prj
* "TIFDistrict.shp" is the name of the shapefile and dallstif is the name of the table I wish to name
* Will be prompted for a password after running inputting this command

== To view all columns of a table ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

pull columns: wkb_geometry, approval_d, expiration

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

pull columns: wkb_geometry, start_, (say that end is 2020)

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Vermont ===

Table name: vermonttif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('vermonttif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid_1 | integer | | | | plain | |
status | character varying | | | | extended | |
tif | character varying | | | | extended | |
acresotv | double precision | | | | plain | |
parcels | integer | | | | plain | |
edtaxablev | integer | | | | plain | |
munitaxabl | integer | | | | plain | |
contactnam | character varying | | | | extended | |
contacttit | character varying | | | | extended | |
contacttel | character varying | | | | extended | |
contactema | character varying | | | | extended | |
stateconta | character varying | | | | extended | |
statetitle | character varying | | | | extended | |
statetelep | character varying | | | | extended | |
stateemail | character varying | | | | extended | |
datelastup | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Dallas, TX ===

Table name: dallastif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('dallastif_gid_seq'::regclass) | plain | |
cvttxdscrp | character varying(150) | | | | extended | |
nofore | bigint | | | | plain | |
noappl | bigint | | | | plain | |
nosale | bigint | | | | plain | |
sumasvlchg | numeric | | | | main | |
sumtxvlchg | numeric | | | | main | |
sumtxodchg | numeric | | | | main | |
lastupdate | date | | | | plain | |
cvttxcd | integer | | | | plain | |
name | character varying(150) | | | | extended | |
created_us | character varying(254) | | | | extended | |
created_da | date | | | | plain | |
last_edite | character varying(254) | | | | extended | |
last_edi_1 | date | | | | plain | |
shape_star | numeric | | | | main | |
shape_stle | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

=== Houston, TX ===

Table name: houstontif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+-----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('houstontif_gid_seq'::regclass) | plain | |
perimeter | numeric | | | | main | |
name | character varying(25) | | | | extended | |
siteno | integer | | | | plain | |
cohgis_coh | numeric | | | | main | |
cohgis_c_1 | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

TIF Project

2017-12-07T16:55:40Z

JeeminS: /* Chicago */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== Command to upload SHP file into a table in database (tigertest) ==
researcher@McNairDBServ:/bulk/Jeemin/Dallas$ shp2pgsql -I "TIFDistrict.shp" dallastif | psql -U postgres -W -d tigertest
* Run this command in a folder with all files for the table, such as files of type .shp, .dbf, .shx, .cpg, and .prj
* "TIFDistrict.shp" is the name of the shapefile and dallstif is the name of the table I wish to name
* Will be prompted for a password after running inputting this command

== To view all columns of a table ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

pull columns: wkb_geometry, approval_d, expiration

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Vermont ===

Table name: vermonttif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('vermonttif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid_1 | integer | | | | plain | |
status | character varying | | | | extended | |
tif | character varying | | | | extended | |
acresotv | double precision | | | | plain | |
parcels | integer | | | | plain | |
edtaxablev | integer | | | | plain | |
munitaxabl | integer | | | | plain | |
contactnam | character varying | | | | extended | |
contacttit | character varying | | | | extended | |
contacttel | character varying | | | | extended | |
contactema | character varying | | | | extended | |
stateconta | character varying | | | | extended | |
statetitle | character varying | | | | extended | |
statetelep | character varying | | | | extended | |
stateemail | character varying | | | | extended | |
datelastup | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Dallas, TX ===

Table name: dallastif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('dallastif_gid_seq'::regclass) | plain | |
cvttxdscrp | character varying(150) | | | | extended | |
nofore | bigint | | | | plain | |
noappl | bigint | | | | plain | |
nosale | bigint | | | | plain | |
sumasvlchg | numeric | | | | main | |
sumtxvlchg | numeric | | | | main | |
sumtxodchg | numeric | | | | main | |
lastupdate | date | | | | plain | |
cvttxcd | integer | | | | plain | |
name | character varying(150) | | | | extended | |
created_us | character varying(254) | | | | extended | |
created_da | date | | | | plain | |
last_edite | character varying(254) | | | | extended | |
last_edi_1 | date | | | | plain | |
shape_star | numeric | | | | main | |
shape_stle | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

=== Houston, TX ===

Table name: houstontif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+-----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('houstontif_gid_seq'::regclass) | plain | |
perimeter | numeric | | | | main | |
name | character varying(25) | | | | extended | |
siteno | integer | | | | plain | |
cohgis_coh | numeric | | | | main | |
cohgis_c_1 | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

TIF Project

2017-12-07T15:25:53Z

JeeminS: /* To view all columns of a tale */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== Command to upload SHP file into a table in database (tigertest) ==
researcher@McNairDBServ:/bulk/Jeemin/Dallas$ shp2pgsql -I "TIFDistrict.shp" dallastif | psql -U postgres -W -d tigertest
* Run this command in a folder with all files for the table, such as files of type .shp, .dbf, .shx, .cpg, and .prj
* "TIFDistrict.shp" is the name of the shapefile and dallstif is the name of the table I wish to name
* Will be prompted for a password after running inputting this command

== To view all columns of a table ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Vermont ===

Table name: vermonttif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('vermonttif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid_1 | integer | | | | plain | |
status | character varying | | | | extended | |
tif | character varying | | | | extended | |
acresotv | double precision | | | | plain | |
parcels | integer | | | | plain | |
edtaxablev | integer | | | | plain | |
munitaxabl | integer | | | | plain | |
contactnam | character varying | | | | extended | |
contacttit | character varying | | | | extended | |
contacttel | character varying | | | | extended | |
contactema | character varying | | | | extended | |
stateconta | character varying | | | | extended | |
statetitle | character varying | | | | extended | |
statetelep | character varying | | | | extended | |
stateemail | character varying | | | | extended | |
datelastup | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Dallas, TX ===

Table name: dallastif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('dallastif_gid_seq'::regclass) | plain | |
cvttxdscrp | character varying(150) | | | | extended | |
nofore | bigint | | | | plain | |
noappl | bigint | | | | plain | |
nosale | bigint | | | | plain | |
sumasvlchg | numeric | | | | main | |
sumtxvlchg | numeric | | | | main | |
sumtxodchg | numeric | | | | main | |
lastupdate | date | | | | plain | |
cvttxcd | integer | | | | plain | |
name | character varying(150) | | | | extended | |
created_us | character varying(254) | | | | extended | |
created_da | date | | | | plain | |
last_edite | character varying(254) | | | | extended | |
last_edi_1 | date | | | | plain | |
shape_star | numeric | | | | main | |
shape_stle | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

=== Houston, TX ===

Table name: houstontif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+-----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('houstontif_gid_seq'::regclass) | plain | |
perimeter | numeric | | | | main | |
name | character varying(25) | | | | extended | |
siteno | integer | | | | plain | |
cohgis_coh | numeric | | | | main | |
cohgis_c_1 | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

Jeemin Sim (Work Log)

2017-12-05T18:24:44Z

JeeminS: /* Fall 2017 */

[[Jeemin Sim]] [[Work Logs]] [[Jeemin Sim (WorkLog)|(log page)]]

===Fall 2017===
<onlyinclude>

2017-12-05:
* Uploaded tables with KML files in TIF folder in bulk(E:) drive
** Allegheny County
** Atlanta
** Chicago
** Columbus, OH
** Dublin, OH
** Vermont
** Washington, D.C.

*Updated documentation for:
** http://mcnair.bakerinstitute.org/wiki/TIF_Project#Uploading_TIF_Data_onto_database_.28tigertest.29

* shp2pgsql needs to be installed to upload Shapefiles to PostgreSQL database
** applies for Houston, TX and Dallas, TX
** DONE

2017-12-04:
* To upload KML file into database with specified table name:
researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
* chicagotif table now resides in tigertest database

2017-11-30:
* Displayed map with TIF districts and startups
* Location:
E:\McNair\Projects\Agglomeration\TIF\McNair_Color_Chicago_TIF_and_startups.png
* Added information and steps to ArcMap/ArcGIS documentation page
* Create a project page for 'Working with POSTGIS' and add instructions for uploading KML file onto POSTGIS
** Command used :
*** Logged in as : researcher@McNairDBServ
/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml
* chicago TIF kml file currently downloaded in tigertest with a table name of Layer0
** Figure out how to change layer name while loading kml file
** Instructions pulled from : http://wiki.wildsong.biz/index.php/Loading_data_into_PostGIS#Loading_data_from_KMZ_files

2017-11-28:
* Created and edited [[ArcMap / ArcGIS Documentation]]
* Plotted points for TIFS and Startups in Chicago in one map.
** Location:
E:\McNair\Projects\Agglomeration\TIF\Jeemin_Chicago_TIF_and_Startups_Attempt1
* Used https://mygeodata.cloud/result to convert from KML file to CSV (which was then saved as txt file to be uploaded onto ArcMap)
* Text files located in Local Disk (C:) Drive

2017-11-27:
*Download Chicago TIF data

2017-11-13:
*

2017-11-09:
* Notes on data downloaded:
** Year 2010-2012 data are based on total population, not 25 yrs or over (case for all other tables)
*** Record appears five times total, with same exact column name
***For exmaple: 'Total; Estimate; High school graduate (includes equivalency)' appears five times, with different values.
* TODO:
** Make Projects page for ACS Data
***[[American Community Survey (ACS) Data]]

2017-11-07:
*Yesterday, narrowed down columns of interest from ACS_S1501_educationattain_2016 table.

Id
Id2
Geography
Total; Estimate; Population 25 years and over
Total; Estimate; Population 25 years and over - High school graduate (includes equivalency)
Total; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Total; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Percent; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Total; Estimate; Population 25 years and over - Associate's degree
Total; Margin of Error; Population 25 years and over - Associate's degree
Percent; Estimate; Population 25 years and over - Associate's degree
Percent; Margin of Error; Population 25 years and over - Associate's degree
Total; Estimate; Population 25 years and over - Bachelor's degree
Total; Margin of Error; Population 25 years and over - Bachelor's degree
Percent; Estimate; Population 25 years and over - Bachelor's degree
Percent; Margin of Error; Population 25 years and over - Bachelor's degree
Total; Estimate; Population 25 years and over - Graduate or professional degree
Total; Margin of Error; Population 25 years and over - Graduate or professional degree
Percent; Estimate; Population 25 years and over - Graduate or professional degree
Percent; Margin of Error; Population 25 years and over - Graduate or professional degree
Percent; Estimate; Percent high school graduate or higher
Percent; Margin of Error; Percent high school graduate or higher
Percent; Estimate; Percent bachelor's degree or higher
Percent; Margin of Error; Percent bachelor's degree or higher

*Complications:
**For csv files corresponding to years 2015 & 2016, all of the above columns exist.
**For csv files corresponding to years 2005 - 2014, no 'Percent' columns exist
*** Instead their 'Total' columns are percentage values
**For csv file corresponding to year 2005, columns regarding Graduate or professional degree are labeled differently.
**2012 data doesn't correspond to Population 25 years and over.

*Temporary Solution:
**Since the above problems may be specific to this set of tables, will go through csv files and adjust columns.

*Python script location:
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data\pullCertainColumns.py

2017-10-31:
* Finished doanloading files from ACS.
* Started loading tables into tigertest.
* Commands run could be found in
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data\DataLoading_SQL_Commands.txt

2017-10-30:
* Downloaded data from ACS, to be continued
* File path:
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data
* Fields of interest:
S1401 SCHOOL ENROLLMENT
S1501 EDUCATIONAL ATTAINMENT
S2301 EMPLOYMENT STATUS
B01003 TOTAL POPULATION
B02001 RACE
B07201 GEOGRAPHICAL MOBILITY
B08303 TRAVEL TIME TO WORK
B19013 MEDIAN HOUSEHOLD INCOME
B19053 SELF-EMPLOYMENT INCOME IN THE PAST 12 MONTHS FOR HOUSEHOLDS
B19083 GINI INDEX OF INCOME INEQUALITY
B25003 TENURE
B25105 MEDIAN MONTHLY HOUSING COSTS
B28011 INTERNET SUBSCRIPTIONS IN HOUSEHOLD
G001 GEOGRAPHIC IDENTIFIERS

2017-10-23:
* Talked to Ed with Peter & Oliver about upcoming tasks & projects.
* Loaded acs_place table 2017 (does not contain population) on tigertest.
** SQL commands used:

DROP TABLE acs_place;

CREATE TABLE acs_place (
USPS varchar(5),
GEOID varchar(30),
ANSICODE varchar(30),
NAME varchar(100),
LSAD varchar(30),
FUNCSTAT varchar(10),
ALAND varchar(30),
AWATER varchar(30),
ALAND_SQMI varchar(30),
AWATER_SQMI varchar(30),
INTPTLAT varchar(30),
INTPTLONG varchar(30)
);

\COPY acs_place FROM '/bulk/2017_Gaz_place_national.txt';
--COPY 29578

* TODO:
** Find acs place 2016 data for population
** Find larger acs files, ideally at the place level
** Provide more documentation on POSTGIS & geocoding

2017-10-16:
* Exported maps of points from the Bay Area each year.
** Map used location: E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS\Jeemin_Bay_Area Points_Every_Year\BayAreaEveryYearMap
** Zoom scale: 1:650.000
* Location of Bay Area Points png files:
E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS\Jeemin_Bay_Area Points_Every_Year

2017-10-10:
*Discoveries/ Struggles regarding.gdb to .shp file conversion:
** Esri Production Mapping (costly)
*** License needs to be purchasesd: http://www.esri.com/software/arcgis/extensions/production-mapping/pricing
** Use ogr2ogr from gdal package
*** https://gis.stackexchange.com/questions/14432/migrating-geodatabase-data-into-postgis-without-esri-apps
*** Command: ogr2ogr -f "ESRI Shapefile" [Destination of shapefile] [path to gdb file]
*** Problem installing gdal

2017-10-09:
* TODO'S:
** Downloading data onto tigertest
*** Road
*** Railway
*** Coastline
*** Instructions: http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation#Bulk_Download_TIGER_Shapefiles
** Configure census data from American Community Survey (ACS)
*** 1) Work out what data is of our interest (confirm ACS)
*** 2) Determine appropriate shape file unit:
**** census block vs. census block group vs. census track
*** 3) Load into tigertest

* Done:
** Downloaded data from https://www.census.gov/cgi-bin/geo/shapefiles/index.php
tl_2017_us_coastline -- 4209
tl_2017_us_primaryroads -- 11574
tl_2017_us_rails -- 176237
** Link found to potentially download ACS data: https://www.census.gov/geo/maps-data/data/tiger-data.html
*** But most files on it come with .gdb extension and not .shp

2017-10-03:
* Installed PostGIS & is now visible on pgAdmin III

* ArcGIS (connect to postgis database):
** 1) Open ArcMap
** 2) Either open blank or open existing file/project
** 3) Click on 'Add Data' button with a cross and a yellow diamond (under Selection toolbar)
** 4) Go to the top-most directory by pressing on the arrow that points left-then-up (on the left of home button)
** 5) Click on 'Database Connections'
** 6) Click on 'Add Database Connection' (if Connection to localhost.sde) does not exist already)
** 7) Fill in the following fields:
*** Database Platform: PostgreSQL
*** Instance: localhost
*** User name: postgres
*** Password:
*** Database: tigertest
** 8) Press 'OK'
** 9) Now you'll have 'Connection to localhost.sde' in your Database Connections
** 10) Double click on 'Connection to localhost.sde'
** 11) Double click on the table of interest
** 12) Click 'Finish'
** 13) You'll see information populated on map, as one of the 'Layers'
*** Tested with: tigertest.public.copointplacescontains

* On running & altering Oliver's script:
** Location: E:\McNair\Projects\OliverLovesCircles\src\python\vc_circles.py
** Ed manipulated file names so that underscores would replace dots (St.Louis --> St_Louis)
** Takes in instances and sweep times as part of the argument, but not impactful as those variables are hardcoded in the script
** Ran vc_circles.py with the following variables with changed values:
*** SWEEP_CYCLE_SECONDS = 10 (used to be 30)
*** NUMBER_INSTANCES = 16 (used to be 8)
** New output to be found in: E:\McNair\Projects\OliverLovesCircles\out

2017-10-02:
* Talked to Harrison & Peter regarding ArcGIS
** Currently have points plotted on Houston
** Trouble interpreting geometry type, as currently reads in from text file
** Documents located in : E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS
* Attempted to install PostGIS spatial extention from PostgreSQL but getting 'spatial database creation failed' error message.
** Referenced instructions:
*** https://www.gpsfiledepot.com/tutorials/installing-and-setting-up-postgresql-with-postgis/
*** http://www.bostongis.com/PrinterFriendly.aspx?content_name=postgis_tut01

2017-09-26:
* Created a table that maps a state to the database name.
** http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation#Translating_Table_names_to_corresponding_States
* Added more GIS-information (functions, realm & outliers to consider)
** http://mcnair.bakerinstitute.org/wiki/Urban_Start-up_Agglomeration#GIS_Resources
* Visualization in PostGIS or connecting to ArcGIS for visualization (import/export data)
* Spatial indexing:
** http://revenant.ca/www/postgis/workshop/indexing.html

2017-09-25:
* Talked to Ed about GIS, Census data, and going about determining the correctness of reported 'place.' Currently script makes a cross product of each reported place and an existing place, outputting a column of boolean value to indicate whether the reported place's coordinates fell within a place's geometric boundaries. One other way of going about this which we discussed is to first check if the reported place does fall within that place's boundaries. If it isn't, we'll go about the cross product method.

* To add documentation :
** http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation
** http://mcnair.bakerinstitute.org/wiki/Urban_Start-up_Agglomeration

* Discussed the need to maintain venture capital database.

*Relevant File paths:
**E:\McNair\Projects\Agglomeration\TestGIS.sql
**Z:\VentureCapitalData\SDCVCData\vcdb2\ProecssingCoLevelSimple.sql
**Z:\VentureCapitalData\SDCVCData\vcdb2\CitiesWithGT10Active.txt

2017-09-21:
* Functions for Linear Referencing:
'''ST_LineInterpolatePoint(geometry A, double measure)''': Returns a point interpolated along a line.
'''ST_LineLocatePoint(geometry A, geometry B)''': Returns a float between 0 and 1 representing the location of the closest point on LineString to the given Point.
'''ST_Line_Substring(geometry A, double from, double to)''': Return a linestring being a substring of the input one starting and ending at the given fractions of total 2d length.
'''ST_Locate_Along_Measure(geometry A, double measure)''': Return a derived geometry collection value with elements that match the specified measure.
'''ST_Locate_Between_Measures(geometry A, double from, double to)''': Return a derived geometry collection value with elements that match the specified range of measures inclusively.
'''ST_AddMeasure(geometry A, double from, double to)''': Return a derived geometry with measure elements linearly interpolated between the start and end points. If the geometry has no measure dimension, one is added.

*3-D Functions:
'''ST_3DClosestPoint''' — Returns the 3-dimensional point on g1 that is closest to g2. This is the first point of the 3D shortest line.
'''ST_3DDistance''' — For geometry type Returns the 3-dimensional cartesian minimum distance (based on spatial ref) between two geometries in projected units.
'''ST_3DDWithin''' — For 3d (z) geometry type Returns true if two geometries 3d distance is within number of units.
'''ST_3DDFullyWithin''' — Returns true if all of the 3D geometries are within the specified distance of one another.
'''ST_3DIntersects''' — Returns TRUE if the Geometries “spatially intersect” in 3d - only for points and linestrings
'''ST_3DLongestLine''' — Returns the 3-dimensional longest line between two geometries
'''ST_3DMaxDistance''' — For geometry type Returns the 3-dimensional cartesian maximum distance (based on spatial ref) between two geometries in projected units.
'''ST_3DShortestLine''' — Returns the 3-dimensional shortest line between two geometries

*Relevant PostgreSQL Commands:
'''\dt *.*''' Show all tables
'''\q''' Exit table

*Specifities/ Outliers to consider:
New York (decompose)
Princeton area (keep Princeton unique)
Reston, Virginia (keep)
San Diego (include La Jolla)
Silicon Valley (all distinct)

* Continue reading from: https://postgis.net/docs/postgis_installation.html

2017-09-20:
* Attended first intro to GIS course yesterday
* Updated above notes on GIS

2017-09-19:
* Useful functions for spatial joins:

'''sum(expression)''': aggregate to return a sum for a set of records
'''count(expression)''': aggregate to return the size of a set of records
'''ST_Area(geometry)''' returns the area of the polygons
'''ST_AsText(geometry)''' returns WKT text
'''ST_Buffer(geometry, distance)''': For geometry: Returns a geometry that represents all points whose distance from this Geometry is less than or equal to distance. Calculations are in the Spatial Reference System of this Geometry. For geography: Uses a planar transform wrapper.
'''ST_Contains(geometry A, geometry B)''' returns the true if geometry A contains geometry B
'''ST_Distance(geometry A, geometry B)''' returns the minimum distance between geometry A and geometry B
'''ST_DWithin(geometry A, geometry B, radius)''' returns the true if geometry A is radius distance or less from geometry B
'''ST_GeomFromText(text)''' returns geometry
'''ST_Intersection(geometry A, geometry B)''': Returns a geometry that represents the shared portion of geomA and geomB. The geography implementation does a transform to geometry to do the intersection and then transform back to WGS84
'''ST_Intersects(geometry A, geometry B)''' returns the true if geometry A intersects geometry B
'''ST_Length(linestring)''' returns the length of the linestring
'''ST_Touches(geometry A, geometry B)''' returns the true if the boundary of geometry A touches geometry B
'''ST_Within(geometry A, geometry B)''' returns the true if geometry A is within geometry B
geometry_a '''&&''' geometry_b: Returns TRUE if A’s bounding box overlaps B’s.
geometry_a '''=''' geometry_b: Returns TRUE if A’s bounding box is the same as B’s.
'''ST_SetSRID(geometry, srid)''': Sets the SRID on a geometry to a particular integer value.
'''ST_SRID(geometry)''': Returns the spatial reference identifier for the ST_Geometry as defined in spatial_ref_sys table.
'''ST_Transform(geometry, srid)''': Returns a new geometry with its coordinates transformed to the SRID referenced by the integer parameter.
'''ST_Union()''': Returns a geometry that represents the point set union of the Geometries.
'''substring(string [from int] [for int])''': PostgreSQL string function to extract substring matching SQL regular expression.
'''ST_Relate(geometry A, geometry B)''': Returns a text string representing the DE9IM relationship between the geometries.
'''ST_GeoHash(geometry A)''': Returns a text string representing the GeoHash of the bounds of the object.

*Native functions for geogrphy:
'''ST_AsText(geography)''' returns text
'''ST_GeographyFromText(text)''' returns geography
'''ST_AsBinary(geography)''' returns bytea
'''ST_GeogFromWKB(bytea)''' returns geography
'''ST_AsSVG(geography)''' returns text
'''ST_AsGML(geography)''' returns text
'''ST_AsKML(geography)''' returns text
'''ST_AsGeoJson(geography)''' returns text
'''ST_Distance(geography, geography)''' returns double
'''ST_DWithin(geography, geography, float8)''' returns boolean
'''ST_Area(geography)''' returns double
'''ST_Length(geography)''' returns double
'''ST_Covers(geography, geography)''' returns boolean
'''ST_CoveredBy(geography, geography)''' returns boolean
'''ST_Intersects(geography, geography)''' returns boolean
'''ST_Buffer(geography, float8)''' returns geography [1]
'''ST_Intersection(geography, geography)''' returns geography [1]
* Continue reading from: http://workshops.boundlessgeo.com/postgis-intro/geography.html

2017-09-18:
* Read documentation on PostGIS and tiger geocoder
* Continue reading from: http://workshops.boundlessgeo.com/postgis-intro/joins.html

2017-09-12:
* Clarified University Matching output file.
* Helped Christy with pdf-reader, capturing keywords in readable format.

2017-09-11:
* Ensured that documentation exists for the projects worked on last semester.

</onlyinclude>

===Spring 2017===

2017-04-17:
* To pull accelerators: Wrote simple python regex-based script that ran on organizations data.
Code: E:\McNair\Projects\Accelerators\Crunchbase Snapshot\accelerator keywords.py
Matched output (885 mathces) : E:\McNair\Projects\Accelerators\Crunchbase Snapshot\Jeemin_885_accel_matches

2017-04-14:
* Loaded Federal Grants Data into database

2017-04-12:
* Finishing up cleaning the columns for Federal Grant Data - NIH. The output excel files can be accessed at:
E:\McNair\Projects\Federal Grant Data\NIH\Grants
Titled:
Jeemin_combined_files 1986-2001.csv
Jeemin_combined_files 2002-2012.csv
Jeemin_combined_files 2013-2015.csv

* psql table formula:

CREATE TABLE all_grants (
APPLICATION_ID integer,
ACTIVITY varchar(3),
ADMINISTERING_IC varchar(2),
APPLICATION_TYPE varchar(1),
ARRA_FUNDED varchar(1),
AWARD_NOTICE_DATE date,
BUDGET_START date,
BUDGET_END date,
CFDA_CODE varchar(3),
CORE_PROJECT_NUM varchar(11),
ED_INST_TYPE varchar(30),
FOA_NUMBER varchar(13),
FULL_PROJECT_NUM varchar(35),
FUNDING_ICs varchar(40),
FUNDING_MECHANISM varchar(23),
FY smallint,
IC_NAME varchar(77),
NIH_SPENDING_CATS varchar(295),
ORG_CITY varchar(20),
ORG_COUNTRY varchar(16),
ORG_DEPT varchar(30),
ORG_DISTRICT smallint,
ORG_DUNS integer,
ORG_FIPS varchar(2),
ORG_NAME varchar(60),
ORG_STATE varchar(2),
ORG_ZIPCODE integer,
PHR varchar(200),
PI_IDS varchar(30),
PI_NAMEs varchar(200),
PROGRAM_OFFICER_NAME varchar(36),
PROJECT_START date,
PROJECT_END date,
PROJECT_TERMS varchar(200),
PROJECT_TITLE varchar(244),
SERIAL_NUMBER smallint,
STUDY_SECTION varchar(4),
STUDY_SECTION_NAME varchar(100),
SUBPROJECT_ID smallint,
SUFFIX varchar(2),
SUPPORT_YEAR smallint,
DIRECT_COST_AMT integer,
INDIRECT_COST_AMT integer,
TOTAL_COST integer,
TOTAL_COST_SUB_PROJECT integer
);
\COPY all_grants FROM 'Jeemin_combined_files 1986-2001.csv' WITH DELIMITER AS E'\t' HEADER NULL AS ''CSV

2017-03-29:
* Troubled by the variety of cases - separating keys by keywords will not work favorably when it hits University of California vs. University of Southern California case - find a way to match University of Southern California first (more specific ones first) - but how to generalize

2017-03-27:
* Writing code for university matches - decided to go through keys instead of each dataitem. Use keywords in each key to go through the dataitem - misspellings are currently unaccounted for.

2017-03-24:
* Discussed with Julia & Meghana about university keys to use to count # of occurrences, including aliases and misspellings
* Thoughts: to use a scoring metric with a key of UNIVERSITY OF CALIFORNIA SYSTEM, it should have a 'better' score when compared to MATHEMATICAL SCIENCES PUBLISHERS C/O UNIVERSITY OF CALIFORNIA BERKELEY or CALIFORNIA AT LOS ANGELES, UNVIERSITY OF than when compared to UNIVERSITY OF SOUTHERN CALIFORNIA, which may pose a challenge when attempting to implement this in a more general sense. In normalizing a string, strip "THE", "," and split words by spaces and compare each keyword from the two strings. Deciding on which strings to compare will be another issue - length (within some range maybe) could be an option.
* Federal Grant Data XML Parser was rerun - same output textfiles

2017-03-22:
* Read string matching & calculating distance, below are relevant links
* [http://www.cs.cmu.edu/~wcohen/postscript/ijcai-ws-2003.pdf]
* [http://web.archive.org/web/20081224234350/http://www.dcs.shef.ac.uk/~sam/stringmetrics.html]

2017-03-22:
* Talked to Julia about universal matcher, want to combine all University of California's to University of California, The Regents of
* Converted crunchbase2013 data from mySQL to PostgreSQL, but having trouble with the last table - cb_relationships, complains about syntax error at or near some places - but generally all tables exist in database called crunchbase
* Federal Grant Data XML Parser was run - the three output textfiles can be found in E:\McNair\Projects\Federal Grant Data\NSF

2017-03-16:
* Further documented [[University Patent Matching]]
* Finished writing XML Parser

2017-03-15:
* Todo: write a wikipage on possible input/output info on string matcher
* Wrote part of XML parser, extracted yearly data into E:\McNair\Projects\Federal Grant Data\NSF\NSF Extracted Data (up to year 2010)

2017-03-14:
* Started pulling academy cases but there are too many cases to worry about, in terms of institution of interest. A document is located in E:\McNair\Projects\University Patents\academies_verify_cases.txt
* Need Julia/Meghana to look through the hits and see which are relevant & extract pattern from there.
* Having trouble outputting txt file without double quotes around every line.
* Thinking that one text file should be output for all keywords instead of having one each, to avoid overlap (ex) COLLEGE and UNIVERSITY are both keywords; ALBERT EINSTEIN COLLEGE OF YESHIVA UNIVERSITY will be hit twice if it were counted as two separate instances, one accounting for COLLEGE and the other for UNIVERSITY) - either in the form of if-elseif statements or one big regex check.

2017-03-13:
* For University Patent Data Matching - matched SCHOOL (output: E:\McNair\Projects\University Patents\school_pulled_from_assignee_list_USA) and matched INSTITUTE(output: E:\McNair\Projects\University Patents\institute_pulled_from_assignee_list_USA).
* [[University Patent Matching]]
* To be worked on later: Grant XML parsing & general name matcher

2017-03-08:
* Wrote regex pattern that identifies all "university" matchings - can be found in E:\McNair\Projects\University Patents\university_pulled_from_assignee_list_USA -- is an output file
* Talked to Sonia, but didn't come to solid conclusion on identifying whether key words associate with city or country by running a python function
* Output sql tables from finished run of Jeemin_FDATrial_as_key_data_ripping.py
* Ran through assigneelist_USA.txt to see how many different ways UNIVERSITY could be spelled wrong. There were many.
* Tried to logic through creating a pattern that could catch all different versions of UNIVERSITY. Discuss further on whether UNIVERSITIES and those that include UNIVERSITIES but include INC in the end should be pulled as relevant information

2017-03-06:
* [[Installing python in a database]]
* Added building Python function section to [[Working with PostgreSQL]] at the bottom of the page.
* Ran FDA Trial data ripping again, as the text output files were wiped.
* Plan on discussing with Julia and Meghana again about pulling universities and other relevant institutions from the Assignee List USA.
* Talked to Sonia about pulling city, state, zipcode information, hence python was installed in a database. Will work with Sonia on Wednesday afternoon and see how best a regex function could be implemented

2017-03-03:
* Attempted to output sql tables

2017-03-01:
* Started re-running Jeemin_FDATrial_as_key_data_ripping.py

2017-02-27:
* Finished producing tables from Jeemin_FDATrial_as_key_data_ripping.py
* Talked to Julia about LinkedIn data extracting - to be discussed further with Julia & Peter.
* Started web crawler for Wikipedia - currently pulls Endowment, Academic staff, students, undergraduates, and postgraduates info found on Rice Wikipedia page. Can be found in : E:\McNair\Projects\University Patents\Jeemin_University_wikipedia_crawler.py

2017-02-24:
* Continued working on producing multiple tables - first two are done. Was working on location, as there are multiple location tags per location.

2017-02-22:
* Finished Jeemin_FDATrial_as_key_data_ripping.py (E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_FDATrial_as_key_data_ripping.py), which outputs to E:\McNair\Projects\FDA Trials\Jeemin_Project\general_data_ripping_output.txt; TODO: output four different tables & replace the write in the same for-loop as going through each file

2017-02-20:
* Continued working on Jeemin_FDATrial_as_key_data_ripping.py to find tags and place all of those information in a list. The other zipcode file did not finish executing after 2+ hours of running it - considering the possibility of splitting the record file into smaller bits, or running the processing on a faster machine.

2017-02-17:
* Completed code for counting the number of occurrences for each unique zipcode. (currently titled & located: E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_Running_File.py). It has been running for 20+min because of the comprehensive XML data files. Meanwhile started coding to create a dictionary with the keys corresponding to each unique trial ID, mapped to every other information (location, sponsors, phase, drugs ...etc.) (currently titled & located: E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_FDATrial_as_key_data_ripping.py).

2017-02-15:
* Discussed with Catherine what to do with FDA Trial data and decided to have a dictionary with zip-codes as keys and number of trials occurred in that zipcode as values. Was still attempting to loop through the files without the code having to exist in the same directory as the XML files. Plan to write to excel via tsv, with zip-code as one column and # of occurrence as the other.

2017-02-13:
* Goals (for trials): 1) Build ER Diagram 2) For each entity, get XML snippet 3) Build a parser/ripper for single file; the python parser can be found at: E:\McNair\Projects\FDA Trials\Jeemin_Project

2017-02-08:
* Attempted to come up with possible cases for locating the description of accelerators - pick up from extracting bodies of text from the about page (given that it exists)
* [[Trial Data Project]]

2017-02-06:
* Set up wikiPage & remote desktop.
* Started working on python version of web crawler. So far it successfully prints out a catchphrase/ description for one website. To be worked on. The python file can be found in: E:\McNair\Projects\Accelerators\Python WebCrawler\webcrawlerpython.py

[[Category:Work Log]]

TIF Project

2017-12-05T18:21:37Z

JeeminS: /* Cities & Tables */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== Command to upload SHP file into a table in database (tigertest) ==
researcher@McNairDBServ:/bulk/Jeemin/Dallas$ shp2pgsql -I "TIFDistrict.shp" dallastif | psql -U postgres -W -d tigertest
* Run this command in a folder with all files for the table, such as files of type .shp, .dbf, .shx, .cpg, and .prj
* "TIFDistrict.shp" is the name of the shapefile and dallstif is the name of the table I wish to name
* Will be prompted for a password after running inputting this command

== To view all columns of a tale ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Vermont ===

Table name: vermonttif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('vermonttif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid_1 | integer | | | | plain | |
status | character varying | | | | extended | |
tif | character varying | | | | extended | |
acresotv | double precision | | | | plain | |
parcels | integer | | | | plain | |
edtaxablev | integer | | | | plain | |
munitaxabl | integer | | | | plain | |
contactnam | character varying | | | | extended | |
contacttit | character varying | | | | extended | |
contacttel | character varying | | | | extended | |
contactema | character varying | | | | extended | |
stateconta | character varying | | | | extended | |
statetitle | character varying | | | | extended | |
statetelep | character varying | | | | extended | |
stateemail | character varying | | | | extended | |
datelastup | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Dallas, TX ===

Table name: dallastif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('dallastif_gid_seq'::regclass) | plain | |
cvttxdscrp | character varying(150) | | | | extended | |
nofore | bigint | | | | plain | |
noappl | bigint | | | | plain | |
nosale | bigint | | | | plain | |
sumasvlchg | numeric | | | | main | |
sumtxvlchg | numeric | | | | main | |
sumtxodchg | numeric | | | | main | |
lastupdate | date | | | | plain | |
cvttxcd | integer | | | | plain | |
name | character varying(150) | | | | extended | |
created_us | character varying(254) | | | | extended | |
created_da | date | | | | plain | |
last_edite | character varying(254) | | | | extended | |
last_edi_1 | date | | | | plain | |
shape_star | numeric | | | | main | |
shape_stle | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

=== Houston, TX ===

Table name: houstontif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+-----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('houstontif_gid_seq'::regclass) | plain | |
perimeter | numeric | | | | main | |
name | character varying(25) | | | | extended | |
siteno | integer | | | | plain | |
cohgis_coh | numeric | | | | main | |
cohgis_c_1 | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

TIF Project

2017-12-05T18:19:46Z

JeeminS: /* Dallas, TX */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== Command to upload SHP file into a table in database (tigertest) ==
researcher@McNairDBServ:/bulk/Jeemin/Dallas$ shp2pgsql -I "TIFDistrict.shp" dallastif | psql -U postgres -W -d tigertest
* Run this command in a folder with all files for the table, such as files of type .shp, .dbf, .shx, .cpg, and .prj
* "TIFDistrict.shp" is the name of the shapefile and dallstif is the name of the table I wish to name
* Will be prompted for a password after running inputting this command

== To view all columns of a tale ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Vermont ===

Table name: vermonttif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('vermonttif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid_1 | integer | | | | plain | |
status | character varying | | | | extended | |
tif | character varying | | | | extended | |
acresotv | double precision | | | | plain | |
parcels | integer | | | | plain | |
edtaxablev | integer | | | | plain | |
munitaxabl | integer | | | | plain | |
contactnam | character varying | | | | extended | |
contacttit | character varying | | | | extended | |
contacttel | character varying | | | | extended | |
contactema | character varying | | | | extended | |
stateconta | character varying | | | | extended | |
statetitle | character varying | | | | extended | |
statetelep | character varying | | | | extended | |
stateemail | character varying | | | | extended | |
datelastup | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Dallas, TX ===

Table name: dallastif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('dallastif_gid_seq'::regclass) | plain | |
cvttxdscrp | character varying(150) | | | | extended | |
nofore | bigint | | | | plain | |
noappl | bigint | | | | plain | |
nosale | bigint | | | | plain | |
sumasvlchg | numeric | | | | main | |
sumtxvlchg | numeric | | | | main | |
sumtxodchg | numeric | | | | main | |
lastupdate | date | | | | plain | |
cvttxcd | integer | | | | plain | |
name | character varying(150) | | | | extended | |
created_us | character varying(254) | | | | extended | |
created_da | date | | | | plain | |
last_edite | character varying(254) | | | | extended | |
last_edi_1 | date | | | | plain | |
shape_star | numeric | | | | main | |
shape_stle | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

TIF Project

2017-12-05T18:19:34Z

JeeminS: /* Cities & Tables */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== Command to upload SHP file into a table in database (tigertest) ==
researcher@McNairDBServ:/bulk/Jeemin/Dallas$ shp2pgsql -I "TIFDistrict.shp" dallastif | psql -U postgres -W -d tigertest
* Run this command in a folder with all files for the table, such as files of type .shp, .dbf, .shx, .cpg, and .prj
* "TIFDistrict.shp" is the name of the shapefile and dallstif is the name of the table I wish to name
* Will be prompted for a password after running inputting this command

== To view all columns of a tale ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Vermont ===

Table name: vermonttif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('vermonttif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid_1 | integer | | | | plain | |
status | character varying | | | | extended | |
tif | character varying | | | | extended | |
acresotv | double precision | | | | plain | |
parcels | integer | | | | plain | |
edtaxablev | integer | | | | plain | |
munitaxabl | integer | | | | plain | |
contactnam | character varying | | | | extended | |
contacttit | character varying | | | | extended | |
contacttel | character varying | | | | extended | |
contactema | character varying | | | | extended | |
stateconta | character varying | | | | extended | |
statetitle | character varying | | | | extended | |
statetelep | character varying | | | | extended | |
stateemail | character varying | | | | extended | |
datelastup | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Dallas, TX ===

Table name: dallastif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
------------+------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
gid | integer | | not null | nextval('dallastif_gid_seq'::regclass) | plain | |
cvttxdscrp | character varying(150) | | | | extended | |
nofore | bigint | | | | plain | |
noappl | bigint | | | | plain | |
nosale | bigint | | | | plain | |
sumasvlchg | numeric | | | | main | |
sumtxvlchg | numeric | | | | main | |
sumtxodchg | numeric | | | | main | |
lastupdate | date | | | | plain | |
cvttxcd | integer | | | | plain | |
name | character varying(150) | | | | extended | |
created_us | character varying(254) | | | | extended | |
created_da | date | | | | plain | |
last_edite | character varying(254) | | | | extended | |
last_edi_1 | date | | | | plain | |
shape_star | numeric | | | | main | |
shape_stle | numeric | | | | main | |
shape_area | numeric | | | | main | |
shape_len | numeric | | | | main | |
geom | geometry(MultiPolygon) | | | | main | |

TIF Project

2017-12-05T18:18:37Z

JeeminS: /* Command to upload SHP file into a table in database (tigertest) */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== Command to upload SHP file into a table in database (tigertest) ==
researcher@McNairDBServ:/bulk/Jeemin/Dallas$ shp2pgsql -I "TIFDistrict.shp" dallastif | psql -U postgres -W -d tigertest
* Run this command in a folder with all files for the table, such as files of type .shp, .dbf, .shx, .cpg, and .prj
* "TIFDistrict.shp" is the name of the shapefile and dallstif is the name of the table I wish to name
* Will be prompted for a password after running inputting this command

== To view all columns of a tale ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Vermont ===

Table name: vermonttif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('vermonttif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid_1 | integer | | | | plain | |
status | character varying | | | | extended | |
tif | character varying | | | | extended | |
acresotv | double precision | | | | plain | |
parcels | integer | | | | plain | |
edtaxablev | integer | | | | plain | |
munitaxabl | integer | | | | plain | |
contactnam | character varying | | | | extended | |
contacttit | character varying | | | | extended | |
contacttel | character varying | | | | extended | |
contactema | character varying | | | | extended | |
stateconta | character varying | | | | extended | |
statetitle | character varying | | | | extended | |
statetelep | character varying | | | | extended | |
stateemail | character varying | | | | extended | |
datelastup | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

TIF Project

2017-12-05T18:18:02Z

JeeminS: /* Command to upload KML file into a table in database (tigertest) */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== Command to upload SHP file into a table in database (tigertest) ==
researcher@McNairDBServ:/bulk/Jeemin/Dallas$ shp2pgsql -I "TIFDistrict.shp" dallastif | psql -U postgres -W -d tigertest
(run this command in a folder with all files for the table, such as files of type .shp, .dbf, .shx, .cpg,
and .prj)
("TIFDistrict.shp" is the name of the shapefile and dallstif is the name of the table I wish to name)
(Will be prompted for a password after running inputting this command)

== To view all columns of a tale ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Vermont ===

Table name: vermonttif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('vermonttif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid_1 | integer | | | | plain | |
status | character varying | | | | extended | |
tif | character varying | | | | extended | |
acresotv | double precision | | | | plain | |
parcels | integer | | | | plain | |
edtaxablev | integer | | | | plain | |
munitaxabl | integer | | | | plain | |
contactnam | character varying | | | | extended | |
contacttit | character varying | | | | extended | |
contacttel | character varying | | | | extended | |
contactema | character varying | | | | extended | |
stateconta | character varying | | | | extended | |
statetitle | character varying | | | | extended | |
statetelep | character varying | | | | extended | |
stateemail | character varying | | | | extended | |
datelastup | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

TIF Project

2017-12-05T18:17:49Z

JeeminS: /* Uploading TIF Data onto database (tigertest) */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== Command to upload SHP file into a table in database (tigertest)
researcher@McNairDBServ:/bulk/Jeemin/Dallas$ shp2pgsql -I "TIFDistrict.shp" dallastif | psql -U postgres -W -d tigertest
(run this command in a folder with all files for the table, such as files of type .shp, .dbf, .shx, .cpg,
and .prj)
("TIFDistrict.shp" is the name of the shapefile and dallstif is the name of the table I wish to name)
(Will be prompted for a password after running inputting this command)

== To view all columns of a tale ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Vermont ===

Table name: vermonttif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('vermonttif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid_1 | integer | | | | plain | |
status | character varying | | | | extended | |
tif | character varying | | | | extended | |
acresotv | double precision | | | | plain | |
parcels | integer | | | | plain | |
edtaxablev | integer | | | | plain | |
munitaxabl | integer | | | | plain | |
contactnam | character varying | | | | extended | |
contacttit | character varying | | | | extended | |
contacttel | character varying | | | | extended | |
contactema | character varying | | | | extended | |
stateconta | character varying | | | | extended | |
statetitle | character varying | | | | extended | |
statetelep | character varying | | | | extended | |
stateemail | character varying | | | | extended | |
datelastup | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

Jeemin Sim (Work Log)

2017-12-05T16:48:34Z

JeeminS: /* Fall 2017 */

[[Jeemin Sim]] [[Work Logs]] [[Jeemin Sim (WorkLog)|(log page)]]

===Fall 2017===
<onlyinclude>

2017-12-05:
* Uploaded tables with KML files in TIF folder in bulk(E:) drive
** Allegheny County
** Atlanta
** Chicago
** Columbus, OH
** Dublin, OH
** Vermont
** Washington, D.C.

*Updated documentation for:
** http://mcnair.bakerinstitute.org/wiki/TIF_Project#Uploading_TIF_Data_onto_database_.28tigertest.29

* shp2pgsql needs to be installed to upload Shapefiles to PostgreSQL database
** applies for Houston, TX and Dallas, TX

2017-12-04:
* To upload KML file into database with specified table name:
researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
* chicagotif table now resides in tigertest database

2017-11-30:
* Displayed map with TIF districts and startups
* Location:
E:\McNair\Projects\Agglomeration\TIF\McNair_Color_Chicago_TIF_and_startups.png
* Added information and steps to ArcMap/ArcGIS documentation page
* Create a project page for 'Working with POSTGIS' and add instructions for uploading KML file onto POSTGIS
** Command used :
*** Logged in as : researcher@McNairDBServ
/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml
* chicago TIF kml file currently downloaded in tigertest with a table name of Layer0
** Figure out how to change layer name while loading kml file
** Instructions pulled from : http://wiki.wildsong.biz/index.php/Loading_data_into_PostGIS#Loading_data_from_KMZ_files

2017-11-28:
* Created and edited [[ArcMap / ArcGIS Documentation]]
* Plotted points for TIFS and Startups in Chicago in one map.
** Location:
E:\McNair\Projects\Agglomeration\TIF\Jeemin_Chicago_TIF_and_Startups_Attempt1
* Used https://mygeodata.cloud/result to convert from KML file to CSV (which was then saved as txt file to be uploaded onto ArcMap)
* Text files located in Local Disk (C:) Drive

2017-11-27:
*Download Chicago TIF data

2017-11-13:
*

2017-11-09:
* Notes on data downloaded:
** Year 2010-2012 data are based on total population, not 25 yrs or over (case for all other tables)
*** Record appears five times total, with same exact column name
***For exmaple: 'Total; Estimate; High school graduate (includes equivalency)' appears five times, with different values.
* TODO:
** Make Projects page for ACS Data
***[[American Community Survey (ACS) Data]]

2017-11-07:
*Yesterday, narrowed down columns of interest from ACS_S1501_educationattain_2016 table.

Id
Id2
Geography
Total; Estimate; Population 25 years and over
Total; Estimate; Population 25 years and over - High school graduate (includes equivalency)
Total; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Total; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Percent; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Total; Estimate; Population 25 years and over - Associate's degree
Total; Margin of Error; Population 25 years and over - Associate's degree
Percent; Estimate; Population 25 years and over - Associate's degree
Percent; Margin of Error; Population 25 years and over - Associate's degree
Total; Estimate; Population 25 years and over - Bachelor's degree
Total; Margin of Error; Population 25 years and over - Bachelor's degree
Percent; Estimate; Population 25 years and over - Bachelor's degree
Percent; Margin of Error; Population 25 years and over - Bachelor's degree
Total; Estimate; Population 25 years and over - Graduate or professional degree
Total; Margin of Error; Population 25 years and over - Graduate or professional degree
Percent; Estimate; Population 25 years and over - Graduate or professional degree
Percent; Margin of Error; Population 25 years and over - Graduate or professional degree
Percent; Estimate; Percent high school graduate or higher
Percent; Margin of Error; Percent high school graduate or higher
Percent; Estimate; Percent bachelor's degree or higher
Percent; Margin of Error; Percent bachelor's degree or higher

*Complications:
**For csv files corresponding to years 2015 & 2016, all of the above columns exist.
**For csv files corresponding to years 2005 - 2014, no 'Percent' columns exist
*** Instead their 'Total' columns are percentage values
**For csv file corresponding to year 2005, columns regarding Graduate or professional degree are labeled differently.
**2012 data doesn't correspond to Population 25 years and over.

*Temporary Solution:
**Since the above problems may be specific to this set of tables, will go through csv files and adjust columns.

*Python script location:
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data\pullCertainColumns.py

2017-10-31:
* Finished doanloading files from ACS.
* Started loading tables into tigertest.
* Commands run could be found in
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data\DataLoading_SQL_Commands.txt

2017-10-30:
* Downloaded data from ACS, to be continued
* File path:
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data
* Fields of interest:
S1401 SCHOOL ENROLLMENT
S1501 EDUCATIONAL ATTAINMENT
S2301 EMPLOYMENT STATUS
B01003 TOTAL POPULATION
B02001 RACE
B07201 GEOGRAPHICAL MOBILITY
B08303 TRAVEL TIME TO WORK
B19013 MEDIAN HOUSEHOLD INCOME
B19053 SELF-EMPLOYMENT INCOME IN THE PAST 12 MONTHS FOR HOUSEHOLDS
B19083 GINI INDEX OF INCOME INEQUALITY
B25003 TENURE
B25105 MEDIAN MONTHLY HOUSING COSTS
B28011 INTERNET SUBSCRIPTIONS IN HOUSEHOLD
G001 GEOGRAPHIC IDENTIFIERS

2017-10-23:
* Talked to Ed with Peter & Oliver about upcoming tasks & projects.
* Loaded acs_place table 2017 (does not contain population) on tigertest.
** SQL commands used:

DROP TABLE acs_place;

CREATE TABLE acs_place (
USPS varchar(5),
GEOID varchar(30),
ANSICODE varchar(30),
NAME varchar(100),
LSAD varchar(30),
FUNCSTAT varchar(10),
ALAND varchar(30),
AWATER varchar(30),
ALAND_SQMI varchar(30),
AWATER_SQMI varchar(30),
INTPTLAT varchar(30),
INTPTLONG varchar(30)
);

\COPY acs_place FROM '/bulk/2017_Gaz_place_national.txt';
--COPY 29578

* TODO:
** Find acs place 2016 data for population
** Find larger acs files, ideally at the place level
** Provide more documentation on POSTGIS & geocoding

2017-10-16:
* Exported maps of points from the Bay Area each year.
** Map used location: E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS\Jeemin_Bay_Area Points_Every_Year\BayAreaEveryYearMap
** Zoom scale: 1:650.000
* Location of Bay Area Points png files:
E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS\Jeemin_Bay_Area Points_Every_Year

2017-10-10:
*Discoveries/ Struggles regarding.gdb to .shp file conversion:
** Esri Production Mapping (costly)
*** License needs to be purchasesd: http://www.esri.com/software/arcgis/extensions/production-mapping/pricing
** Use ogr2ogr from gdal package
*** https://gis.stackexchange.com/questions/14432/migrating-geodatabase-data-into-postgis-without-esri-apps
*** Command: ogr2ogr -f "ESRI Shapefile" [Destination of shapefile] [path to gdb file]
*** Problem installing gdal

2017-10-09:
* TODO'S:
** Downloading data onto tigertest
*** Road
*** Railway
*** Coastline
*** Instructions: http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation#Bulk_Download_TIGER_Shapefiles
** Configure census data from American Community Survey (ACS)
*** 1) Work out what data is of our interest (confirm ACS)
*** 2) Determine appropriate shape file unit:
**** census block vs. census block group vs. census track
*** 3) Load into tigertest

* Done:
** Downloaded data from https://www.census.gov/cgi-bin/geo/shapefiles/index.php
tl_2017_us_coastline -- 4209
tl_2017_us_primaryroads -- 11574
tl_2017_us_rails -- 176237
** Link found to potentially download ACS data: https://www.census.gov/geo/maps-data/data/tiger-data.html
*** But most files on it come with .gdb extension and not .shp

2017-10-03:
* Installed PostGIS & is now visible on pgAdmin III

* ArcGIS (connect to postgis database):
** 1) Open ArcMap
** 2) Either open blank or open existing file/project
** 3) Click on 'Add Data' button with a cross and a yellow diamond (under Selection toolbar)
** 4) Go to the top-most directory by pressing on the arrow that points left-then-up (on the left of home button)
** 5) Click on 'Database Connections'
** 6) Click on 'Add Database Connection' (if Connection to localhost.sde) does not exist already)
** 7) Fill in the following fields:
*** Database Platform: PostgreSQL
*** Instance: localhost
*** User name: postgres
*** Password:
*** Database: tigertest
** 8) Press 'OK'
** 9) Now you'll have 'Connection to localhost.sde' in your Database Connections
** 10) Double click on 'Connection to localhost.sde'
** 11) Double click on the table of interest
** 12) Click 'Finish'
** 13) You'll see information populated on map, as one of the 'Layers'
*** Tested with: tigertest.public.copointplacescontains

* On running & altering Oliver's script:
** Location: E:\McNair\Projects\OliverLovesCircles\src\python\vc_circles.py
** Ed manipulated file names so that underscores would replace dots (St.Louis --> St_Louis)
** Takes in instances and sweep times as part of the argument, but not impactful as those variables are hardcoded in the script
** Ran vc_circles.py with the following variables with changed values:
*** SWEEP_CYCLE_SECONDS = 10 (used to be 30)
*** NUMBER_INSTANCES = 16 (used to be 8)
** New output to be found in: E:\McNair\Projects\OliverLovesCircles\out

2017-10-02:
* Talked to Harrison & Peter regarding ArcGIS
** Currently have points plotted on Houston
** Trouble interpreting geometry type, as currently reads in from text file
** Documents located in : E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS
* Attempted to install PostGIS spatial extention from PostgreSQL but getting 'spatial database creation failed' error message.
** Referenced instructions:
*** https://www.gpsfiledepot.com/tutorials/installing-and-setting-up-postgresql-with-postgis/
*** http://www.bostongis.com/PrinterFriendly.aspx?content_name=postgis_tut01

2017-09-26:
* Created a table that maps a state to the database name.
** http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation#Translating_Table_names_to_corresponding_States
* Added more GIS-information (functions, realm & outliers to consider)
** http://mcnair.bakerinstitute.org/wiki/Urban_Start-up_Agglomeration#GIS_Resources
* Visualization in PostGIS or connecting to ArcGIS for visualization (import/export data)
* Spatial indexing:
** http://revenant.ca/www/postgis/workshop/indexing.html

2017-09-25:
* Talked to Ed about GIS, Census data, and going about determining the correctness of reported 'place.' Currently script makes a cross product of each reported place and an existing place, outputting a column of boolean value to indicate whether the reported place's coordinates fell within a place's geometric boundaries. One other way of going about this which we discussed is to first check if the reported place does fall within that place's boundaries. If it isn't, we'll go about the cross product method.

* To add documentation :
** http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation
** http://mcnair.bakerinstitute.org/wiki/Urban_Start-up_Agglomeration

* Discussed the need to maintain venture capital database.

*Relevant File paths:
**E:\McNair\Projects\Agglomeration\TestGIS.sql
**Z:\VentureCapitalData\SDCVCData\vcdb2\ProecssingCoLevelSimple.sql
**Z:\VentureCapitalData\SDCVCData\vcdb2\CitiesWithGT10Active.txt

2017-09-21:
* Functions for Linear Referencing:
'''ST_LineInterpolatePoint(geometry A, double measure)''': Returns a point interpolated along a line.
'''ST_LineLocatePoint(geometry A, geometry B)''': Returns a float between 0 and 1 representing the location of the closest point on LineString to the given Point.
'''ST_Line_Substring(geometry A, double from, double to)''': Return a linestring being a substring of the input one starting and ending at the given fractions of total 2d length.
'''ST_Locate_Along_Measure(geometry A, double measure)''': Return a derived geometry collection value with elements that match the specified measure.
'''ST_Locate_Between_Measures(geometry A, double from, double to)''': Return a derived geometry collection value with elements that match the specified range of measures inclusively.
'''ST_AddMeasure(geometry A, double from, double to)''': Return a derived geometry with measure elements linearly interpolated between the start and end points. If the geometry has no measure dimension, one is added.

*3-D Functions:
'''ST_3DClosestPoint''' — Returns the 3-dimensional point on g1 that is closest to g2. This is the first point of the 3D shortest line.
'''ST_3DDistance''' — For geometry type Returns the 3-dimensional cartesian minimum distance (based on spatial ref) between two geometries in projected units.
'''ST_3DDWithin''' — For 3d (z) geometry type Returns true if two geometries 3d distance is within number of units.
'''ST_3DDFullyWithin''' — Returns true if all of the 3D geometries are within the specified distance of one another.
'''ST_3DIntersects''' — Returns TRUE if the Geometries “spatially intersect” in 3d - only for points and linestrings
'''ST_3DLongestLine''' — Returns the 3-dimensional longest line between two geometries
'''ST_3DMaxDistance''' — For geometry type Returns the 3-dimensional cartesian maximum distance (based on spatial ref) between two geometries in projected units.
'''ST_3DShortestLine''' — Returns the 3-dimensional shortest line between two geometries

*Relevant PostgreSQL Commands:
'''\dt *.*''' Show all tables
'''\q''' Exit table

*Specifities/ Outliers to consider:
New York (decompose)
Princeton area (keep Princeton unique)
Reston, Virginia (keep)
San Diego (include La Jolla)
Silicon Valley (all distinct)

* Continue reading from: https://postgis.net/docs/postgis_installation.html

2017-09-20:
* Attended first intro to GIS course yesterday
* Updated above notes on GIS

2017-09-19:
* Useful functions for spatial joins:

'''sum(expression)''': aggregate to return a sum for a set of records
'''count(expression)''': aggregate to return the size of a set of records
'''ST_Area(geometry)''' returns the area of the polygons
'''ST_AsText(geometry)''' returns WKT text
'''ST_Buffer(geometry, distance)''': For geometry: Returns a geometry that represents all points whose distance from this Geometry is less than or equal to distance. Calculations are in the Spatial Reference System of this Geometry. For geography: Uses a planar transform wrapper.
'''ST_Contains(geometry A, geometry B)''' returns the true if geometry A contains geometry B
'''ST_Distance(geometry A, geometry B)''' returns the minimum distance between geometry A and geometry B
'''ST_DWithin(geometry A, geometry B, radius)''' returns the true if geometry A is radius distance or less from geometry B
'''ST_GeomFromText(text)''' returns geometry
'''ST_Intersection(geometry A, geometry B)''': Returns a geometry that represents the shared portion of geomA and geomB. The geography implementation does a transform to geometry to do the intersection and then transform back to WGS84
'''ST_Intersects(geometry A, geometry B)''' returns the true if geometry A intersects geometry B
'''ST_Length(linestring)''' returns the length of the linestring
'''ST_Touches(geometry A, geometry B)''' returns the true if the boundary of geometry A touches geometry B
'''ST_Within(geometry A, geometry B)''' returns the true if geometry A is within geometry B
geometry_a '''&&''' geometry_b: Returns TRUE if A’s bounding box overlaps B’s.
geometry_a '''=''' geometry_b: Returns TRUE if A’s bounding box is the same as B’s.
'''ST_SetSRID(geometry, srid)''': Sets the SRID on a geometry to a particular integer value.
'''ST_SRID(geometry)''': Returns the spatial reference identifier for the ST_Geometry as defined in spatial_ref_sys table.
'''ST_Transform(geometry, srid)''': Returns a new geometry with its coordinates transformed to the SRID referenced by the integer parameter.
'''ST_Union()''': Returns a geometry that represents the point set union of the Geometries.
'''substring(string [from int] [for int])''': PostgreSQL string function to extract substring matching SQL regular expression.
'''ST_Relate(geometry A, geometry B)''': Returns a text string representing the DE9IM relationship between the geometries.
'''ST_GeoHash(geometry A)''': Returns a text string representing the GeoHash of the bounds of the object.

*Native functions for geogrphy:
'''ST_AsText(geography)''' returns text
'''ST_GeographyFromText(text)''' returns geography
'''ST_AsBinary(geography)''' returns bytea
'''ST_GeogFromWKB(bytea)''' returns geography
'''ST_AsSVG(geography)''' returns text
'''ST_AsGML(geography)''' returns text
'''ST_AsKML(geography)''' returns text
'''ST_AsGeoJson(geography)''' returns text
'''ST_Distance(geography, geography)''' returns double
'''ST_DWithin(geography, geography, float8)''' returns boolean
'''ST_Area(geography)''' returns double
'''ST_Length(geography)''' returns double
'''ST_Covers(geography, geography)''' returns boolean
'''ST_CoveredBy(geography, geography)''' returns boolean
'''ST_Intersects(geography, geography)''' returns boolean
'''ST_Buffer(geography, float8)''' returns geography [1]
'''ST_Intersection(geography, geography)''' returns geography [1]
* Continue reading from: http://workshops.boundlessgeo.com/postgis-intro/geography.html

2017-09-18:
* Read documentation on PostGIS and tiger geocoder
* Continue reading from: http://workshops.boundlessgeo.com/postgis-intro/joins.html

2017-09-12:
* Clarified University Matching output file.
* Helped Christy with pdf-reader, capturing keywords in readable format.

2017-09-11:
* Ensured that documentation exists for the projects worked on last semester.

</onlyinclude>

===Spring 2017===

2017-04-17:
* To pull accelerators: Wrote simple python regex-based script that ran on organizations data.
Code: E:\McNair\Projects\Accelerators\Crunchbase Snapshot\accelerator keywords.py
Matched output (885 mathces) : E:\McNair\Projects\Accelerators\Crunchbase Snapshot\Jeemin_885_accel_matches

2017-04-14:
* Loaded Federal Grants Data into database

2017-04-12:
* Finishing up cleaning the columns for Federal Grant Data - NIH. The output excel files can be accessed at:
E:\McNair\Projects\Federal Grant Data\NIH\Grants
Titled:
Jeemin_combined_files 1986-2001.csv
Jeemin_combined_files 2002-2012.csv
Jeemin_combined_files 2013-2015.csv

* psql table formula:

CREATE TABLE all_grants (
APPLICATION_ID integer,
ACTIVITY varchar(3),
ADMINISTERING_IC varchar(2),
APPLICATION_TYPE varchar(1),
ARRA_FUNDED varchar(1),
AWARD_NOTICE_DATE date,
BUDGET_START date,
BUDGET_END date,
CFDA_CODE varchar(3),
CORE_PROJECT_NUM varchar(11),
ED_INST_TYPE varchar(30),
FOA_NUMBER varchar(13),
FULL_PROJECT_NUM varchar(35),
FUNDING_ICs varchar(40),
FUNDING_MECHANISM varchar(23),
FY smallint,
IC_NAME varchar(77),
NIH_SPENDING_CATS varchar(295),
ORG_CITY varchar(20),
ORG_COUNTRY varchar(16),
ORG_DEPT varchar(30),
ORG_DISTRICT smallint,
ORG_DUNS integer,
ORG_FIPS varchar(2),
ORG_NAME varchar(60),
ORG_STATE varchar(2),
ORG_ZIPCODE integer,
PHR varchar(200),
PI_IDS varchar(30),
PI_NAMEs varchar(200),
PROGRAM_OFFICER_NAME varchar(36),
PROJECT_START date,
PROJECT_END date,
PROJECT_TERMS varchar(200),
PROJECT_TITLE varchar(244),
SERIAL_NUMBER smallint,
STUDY_SECTION varchar(4),
STUDY_SECTION_NAME varchar(100),
SUBPROJECT_ID smallint,
SUFFIX varchar(2),
SUPPORT_YEAR smallint,
DIRECT_COST_AMT integer,
INDIRECT_COST_AMT integer,
TOTAL_COST integer,
TOTAL_COST_SUB_PROJECT integer
);
\COPY all_grants FROM 'Jeemin_combined_files 1986-2001.csv' WITH DELIMITER AS E'\t' HEADER NULL AS ''CSV

2017-03-29:
* Troubled by the variety of cases - separating keys by keywords will not work favorably when it hits University of California vs. University of Southern California case - find a way to match University of Southern California first (more specific ones first) - but how to generalize

2017-03-27:
* Writing code for university matches - decided to go through keys instead of each dataitem. Use keywords in each key to go through the dataitem - misspellings are currently unaccounted for.

2017-03-24:
* Discussed with Julia & Meghana about university keys to use to count # of occurrences, including aliases and misspellings
* Thoughts: to use a scoring metric with a key of UNIVERSITY OF CALIFORNIA SYSTEM, it should have a 'better' score when compared to MATHEMATICAL SCIENCES PUBLISHERS C/O UNIVERSITY OF CALIFORNIA BERKELEY or CALIFORNIA AT LOS ANGELES, UNVIERSITY OF than when compared to UNIVERSITY OF SOUTHERN CALIFORNIA, which may pose a challenge when attempting to implement this in a more general sense. In normalizing a string, strip "THE", "," and split words by spaces and compare each keyword from the two strings. Deciding on which strings to compare will be another issue - length (within some range maybe) could be an option.
* Federal Grant Data XML Parser was rerun - same output textfiles

2017-03-22:
* Read string matching & calculating distance, below are relevant links
* [http://www.cs.cmu.edu/~wcohen/postscript/ijcai-ws-2003.pdf]
* [http://web.archive.org/web/20081224234350/http://www.dcs.shef.ac.uk/~sam/stringmetrics.html]

2017-03-22:
* Talked to Julia about universal matcher, want to combine all University of California's to University of California, The Regents of
* Converted crunchbase2013 data from mySQL to PostgreSQL, but having trouble with the last table - cb_relationships, complains about syntax error at or near some places - but generally all tables exist in database called crunchbase
* Federal Grant Data XML Parser was run - the three output textfiles can be found in E:\McNair\Projects\Federal Grant Data\NSF

2017-03-16:
* Further documented [[University Patent Matching]]
* Finished writing XML Parser

2017-03-15:
* Todo: write a wikipage on possible input/output info on string matcher
* Wrote part of XML parser, extracted yearly data into E:\McNair\Projects\Federal Grant Data\NSF\NSF Extracted Data (up to year 2010)

2017-03-14:
* Started pulling academy cases but there are too many cases to worry about, in terms of institution of interest. A document is located in E:\McNair\Projects\University Patents\academies_verify_cases.txt
* Need Julia/Meghana to look through the hits and see which are relevant & extract pattern from there.
* Having trouble outputting txt file without double quotes around every line.
* Thinking that one text file should be output for all keywords instead of having one each, to avoid overlap (ex) COLLEGE and UNIVERSITY are both keywords; ALBERT EINSTEIN COLLEGE OF YESHIVA UNIVERSITY will be hit twice if it were counted as two separate instances, one accounting for COLLEGE and the other for UNIVERSITY) - either in the form of if-elseif statements or one big regex check.

2017-03-13:
* For University Patent Data Matching - matched SCHOOL (output: E:\McNair\Projects\University Patents\school_pulled_from_assignee_list_USA) and matched INSTITUTE(output: E:\McNair\Projects\University Patents\institute_pulled_from_assignee_list_USA).
* [[University Patent Matching]]
* To be worked on later: Grant XML parsing & general name matcher

2017-03-08:
* Wrote regex pattern that identifies all "university" matchings - can be found in E:\McNair\Projects\University Patents\university_pulled_from_assignee_list_USA -- is an output file
* Talked to Sonia, but didn't come to solid conclusion on identifying whether key words associate with city or country by running a python function
* Output sql tables from finished run of Jeemin_FDATrial_as_key_data_ripping.py
* Ran through assigneelist_USA.txt to see how many different ways UNIVERSITY could be spelled wrong. There were many.
* Tried to logic through creating a pattern that could catch all different versions of UNIVERSITY. Discuss further on whether UNIVERSITIES and those that include UNIVERSITIES but include INC in the end should be pulled as relevant information

2017-03-06:
* [[Installing python in a database]]
* Added building Python function section to [[Working with PostgreSQL]] at the bottom of the page.
* Ran FDA Trial data ripping again, as the text output files were wiped.
* Plan on discussing with Julia and Meghana again about pulling universities and other relevant institutions from the Assignee List USA.
* Talked to Sonia about pulling city, state, zipcode information, hence python was installed in a database. Will work with Sonia on Wednesday afternoon and see how best a regex function could be implemented

2017-03-03:
* Attempted to output sql tables

2017-03-01:
* Started re-running Jeemin_FDATrial_as_key_data_ripping.py

2017-02-27:
* Finished producing tables from Jeemin_FDATrial_as_key_data_ripping.py
* Talked to Julia about LinkedIn data extracting - to be discussed further with Julia & Peter.
* Started web crawler for Wikipedia - currently pulls Endowment, Academic staff, students, undergraduates, and postgraduates info found on Rice Wikipedia page. Can be found in : E:\McNair\Projects\University Patents\Jeemin_University_wikipedia_crawler.py

2017-02-24:
* Continued working on producing multiple tables - first two are done. Was working on location, as there are multiple location tags per location.

2017-02-22:
* Finished Jeemin_FDATrial_as_key_data_ripping.py (E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_FDATrial_as_key_data_ripping.py), which outputs to E:\McNair\Projects\FDA Trials\Jeemin_Project\general_data_ripping_output.txt; TODO: output four different tables & replace the write in the same for-loop as going through each file

2017-02-20:
* Continued working on Jeemin_FDATrial_as_key_data_ripping.py to find tags and place all of those information in a list. The other zipcode file did not finish executing after 2+ hours of running it - considering the possibility of splitting the record file into smaller bits, or running the processing on a faster machine.

2017-02-17:
* Completed code for counting the number of occurrences for each unique zipcode. (currently titled & located: E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_Running_File.py). It has been running for 20+min because of the comprehensive XML data files. Meanwhile started coding to create a dictionary with the keys corresponding to each unique trial ID, mapped to every other information (location, sponsors, phase, drugs ...etc.) (currently titled & located: E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_FDATrial_as_key_data_ripping.py).

2017-02-15:
* Discussed with Catherine what to do with FDA Trial data and decided to have a dictionary with zip-codes as keys and number of trials occurred in that zipcode as values. Was still attempting to loop through the files without the code having to exist in the same directory as the XML files. Plan to write to excel via tsv, with zip-code as one column and # of occurrence as the other.

2017-02-13:
* Goals (for trials): 1) Build ER Diagram 2) For each entity, get XML snippet 3) Build a parser/ripper for single file; the python parser can be found at: E:\McNair\Projects\FDA Trials\Jeemin_Project

2017-02-08:
* Attempted to come up with possible cases for locating the description of accelerators - pick up from extracting bodies of text from the about page (given that it exists)
* [[Trial Data Project]]

2017-02-06:
* Set up wikiPage & remote desktop.
* Started working on python version of web crawler. So far it successfully prints out a catchphrase/ description for one website. To be worked on. The python file can be found in: E:\McNair\Projects\Accelerators\Python WebCrawler\webcrawlerpython.py

[[Category:Work Log]]

Jeemin Sim (Work Log)

2017-12-05T16:47:25Z

JeeminS: /* Fall 2017 */

[[Jeemin Sim]] [[Work Logs]] [[Jeemin Sim (WorkLog)|(log page)]]

===Fall 2017===
<onlyinclude>

2017-12-05:
* Uploaded tables with KML files in TIF folder in bulk(E:) drive
** Allegheny County
** Atlanta
** Chicago
** Columbus, OH
** Dublin, OH
** Vermont
** Washington, D.C.

*Updated documentation for:
** http://mcnair.bakerinstitute.org/wiki/TIF_Project#Uploading_TIF_Data_onto_database_.28tigertest.29

2017-12-04:
* To upload KML file into database with specified table name:
researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
* chicagotif table now resides in tigertest database

2017-11-30:
* Displayed map with TIF districts and startups
* Location:
E:\McNair\Projects\Agglomeration\TIF\McNair_Color_Chicago_TIF_and_startups.png
* Added information and steps to ArcMap/ArcGIS documentation page
* Create a project page for 'Working with POSTGIS' and add instructions for uploading KML file onto POSTGIS
** Command used :
*** Logged in as : researcher@McNairDBServ
/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml
* chicago TIF kml file currently downloaded in tigertest with a table name of Layer0
** Figure out how to change layer name while loading kml file
** Instructions pulled from : http://wiki.wildsong.biz/index.php/Loading_data_into_PostGIS#Loading_data_from_KMZ_files

2017-11-28:
* Created and edited [[ArcMap / ArcGIS Documentation]]
* Plotted points for TIFS and Startups in Chicago in one map.
** Location:
E:\McNair\Projects\Agglomeration\TIF\Jeemin_Chicago_TIF_and_Startups_Attempt1
* Used https://mygeodata.cloud/result to convert from KML file to CSV (which was then saved as txt file to be uploaded onto ArcMap)
* Text files located in Local Disk (C:) Drive

2017-11-27:
*Download Chicago TIF data

2017-11-13:
*

2017-11-09:
* Notes on data downloaded:
** Year 2010-2012 data are based on total population, not 25 yrs or over (case for all other tables)
*** Record appears five times total, with same exact column name
***For exmaple: 'Total; Estimate; High school graduate (includes equivalency)' appears five times, with different values.
* TODO:
** Make Projects page for ACS Data
***[[American Community Survey (ACS) Data]]

2017-11-07:
*Yesterday, narrowed down columns of interest from ACS_S1501_educationattain_2016 table.

Id
Id2
Geography
Total; Estimate; Population 25 years and over
Total; Estimate; Population 25 years and over - High school graduate (includes equivalency)
Total; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Total; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Percent; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Total; Estimate; Population 25 years and over - Associate's degree
Total; Margin of Error; Population 25 years and over - Associate's degree
Percent; Estimate; Population 25 years and over - Associate's degree
Percent; Margin of Error; Population 25 years and over - Associate's degree
Total; Estimate; Population 25 years and over - Bachelor's degree
Total; Margin of Error; Population 25 years and over - Bachelor's degree
Percent; Estimate; Population 25 years and over - Bachelor's degree
Percent; Margin of Error; Population 25 years and over - Bachelor's degree
Total; Estimate; Population 25 years and over - Graduate or professional degree
Total; Margin of Error; Population 25 years and over - Graduate or professional degree
Percent; Estimate; Population 25 years and over - Graduate or professional degree
Percent; Margin of Error; Population 25 years and over - Graduate or professional degree
Percent; Estimate; Percent high school graduate or higher
Percent; Margin of Error; Percent high school graduate or higher
Percent; Estimate; Percent bachelor's degree or higher
Percent; Margin of Error; Percent bachelor's degree or higher

*Complications:
**For csv files corresponding to years 2015 & 2016, all of the above columns exist.
**For csv files corresponding to years 2005 - 2014, no 'Percent' columns exist
*** Instead their 'Total' columns are percentage values
**For csv file corresponding to year 2005, columns regarding Graduate or professional degree are labeled differently.
**2012 data doesn't correspond to Population 25 years and over.

*Temporary Solution:
**Since the above problems may be specific to this set of tables, will go through csv files and adjust columns.

*Python script location:
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data\pullCertainColumns.py

2017-10-31:
* Finished doanloading files from ACS.
* Started loading tables into tigertest.
* Commands run could be found in
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data\DataLoading_SQL_Commands.txt

2017-10-30:
* Downloaded data from ACS, to be continued
* File path:
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data
* Fields of interest:
S1401 SCHOOL ENROLLMENT
S1501 EDUCATIONAL ATTAINMENT
S2301 EMPLOYMENT STATUS
B01003 TOTAL POPULATION
B02001 RACE
B07201 GEOGRAPHICAL MOBILITY
B08303 TRAVEL TIME TO WORK
B19013 MEDIAN HOUSEHOLD INCOME
B19053 SELF-EMPLOYMENT INCOME IN THE PAST 12 MONTHS FOR HOUSEHOLDS
B19083 GINI INDEX OF INCOME INEQUALITY
B25003 TENURE
B25105 MEDIAN MONTHLY HOUSING COSTS
B28011 INTERNET SUBSCRIPTIONS IN HOUSEHOLD
G001 GEOGRAPHIC IDENTIFIERS

2017-10-23:
* Talked to Ed with Peter & Oliver about upcoming tasks & projects.
* Loaded acs_place table 2017 (does not contain population) on tigertest.
** SQL commands used:

DROP TABLE acs_place;

CREATE TABLE acs_place (
USPS varchar(5),
GEOID varchar(30),
ANSICODE varchar(30),
NAME varchar(100),
LSAD varchar(30),
FUNCSTAT varchar(10),
ALAND varchar(30),
AWATER varchar(30),
ALAND_SQMI varchar(30),
AWATER_SQMI varchar(30),
INTPTLAT varchar(30),
INTPTLONG varchar(30)
);

\COPY acs_place FROM '/bulk/2017_Gaz_place_national.txt';
--COPY 29578

* TODO:
** Find acs place 2016 data for population
** Find larger acs files, ideally at the place level
** Provide more documentation on POSTGIS & geocoding

2017-10-16:
* Exported maps of points from the Bay Area each year.
** Map used location: E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS\Jeemin_Bay_Area Points_Every_Year\BayAreaEveryYearMap
** Zoom scale: 1:650.000
* Location of Bay Area Points png files:
E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS\Jeemin_Bay_Area Points_Every_Year

2017-10-10:
*Discoveries/ Struggles regarding.gdb to .shp file conversion:
** Esri Production Mapping (costly)
*** License needs to be purchasesd: http://www.esri.com/software/arcgis/extensions/production-mapping/pricing
** Use ogr2ogr from gdal package
*** https://gis.stackexchange.com/questions/14432/migrating-geodatabase-data-into-postgis-without-esri-apps
*** Command: ogr2ogr -f "ESRI Shapefile" [Destination of shapefile] [path to gdb file]
*** Problem installing gdal

2017-10-09:
* TODO'S:
** Downloading data onto tigertest
*** Road
*** Railway
*** Coastline
*** Instructions: http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation#Bulk_Download_TIGER_Shapefiles
** Configure census data from American Community Survey (ACS)
*** 1) Work out what data is of our interest (confirm ACS)
*** 2) Determine appropriate shape file unit:
**** census block vs. census block group vs. census track
*** 3) Load into tigertest

* Done:
** Downloaded data from https://www.census.gov/cgi-bin/geo/shapefiles/index.php
tl_2017_us_coastline -- 4209
tl_2017_us_primaryroads -- 11574
tl_2017_us_rails -- 176237
** Link found to potentially download ACS data: https://www.census.gov/geo/maps-data/data/tiger-data.html
*** But most files on it come with .gdb extension and not .shp

2017-10-03:
* Installed PostGIS & is now visible on pgAdmin III

* ArcGIS (connect to postgis database):
** 1) Open ArcMap
** 2) Either open blank or open existing file/project
** 3) Click on 'Add Data' button with a cross and a yellow diamond (under Selection toolbar)
** 4) Go to the top-most directory by pressing on the arrow that points left-then-up (on the left of home button)
** 5) Click on 'Database Connections'
** 6) Click on 'Add Database Connection' (if Connection to localhost.sde) does not exist already)
** 7) Fill in the following fields:
*** Database Platform: PostgreSQL
*** Instance: localhost
*** User name: postgres
*** Password:
*** Database: tigertest
** 8) Press 'OK'
** 9) Now you'll have 'Connection to localhost.sde' in your Database Connections
** 10) Double click on 'Connection to localhost.sde'
** 11) Double click on the table of interest
** 12) Click 'Finish'
** 13) You'll see information populated on map, as one of the 'Layers'
*** Tested with: tigertest.public.copointplacescontains

* On running & altering Oliver's script:
** Location: E:\McNair\Projects\OliverLovesCircles\src\python\vc_circles.py
** Ed manipulated file names so that underscores would replace dots (St.Louis --> St_Louis)
** Takes in instances and sweep times as part of the argument, but not impactful as those variables are hardcoded in the script
** Ran vc_circles.py with the following variables with changed values:
*** SWEEP_CYCLE_SECONDS = 10 (used to be 30)
*** NUMBER_INSTANCES = 16 (used to be 8)
** New output to be found in: E:\McNair\Projects\OliverLovesCircles\out

2017-10-02:
* Talked to Harrison & Peter regarding ArcGIS
** Currently have points plotted on Houston
** Trouble interpreting geometry type, as currently reads in from text file
** Documents located in : E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS
* Attempted to install PostGIS spatial extention from PostgreSQL but getting 'spatial database creation failed' error message.
** Referenced instructions:
*** https://www.gpsfiledepot.com/tutorials/installing-and-setting-up-postgresql-with-postgis/
*** http://www.bostongis.com/PrinterFriendly.aspx?content_name=postgis_tut01

2017-09-26:
* Created a table that maps a state to the database name.
** http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation#Translating_Table_names_to_corresponding_States
* Added more GIS-information (functions, realm & outliers to consider)
** http://mcnair.bakerinstitute.org/wiki/Urban_Start-up_Agglomeration#GIS_Resources
* Visualization in PostGIS or connecting to ArcGIS for visualization (import/export data)
* Spatial indexing:
** http://revenant.ca/www/postgis/workshop/indexing.html

2017-09-25:
* Talked to Ed about GIS, Census data, and going about determining the correctness of reported 'place.' Currently script makes a cross product of each reported place and an existing place, outputting a column of boolean value to indicate whether the reported place's coordinates fell within a place's geometric boundaries. One other way of going about this which we discussed is to first check if the reported place does fall within that place's boundaries. If it isn't, we'll go about the cross product method.

* To add documentation :
** http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation
** http://mcnair.bakerinstitute.org/wiki/Urban_Start-up_Agglomeration

* Discussed the need to maintain venture capital database.

*Relevant File paths:
**E:\McNair\Projects\Agglomeration\TestGIS.sql
**Z:\VentureCapitalData\SDCVCData\vcdb2\ProecssingCoLevelSimple.sql
**Z:\VentureCapitalData\SDCVCData\vcdb2\CitiesWithGT10Active.txt

2017-09-21:
* Functions for Linear Referencing:
'''ST_LineInterpolatePoint(geometry A, double measure)''': Returns a point interpolated along a line.
'''ST_LineLocatePoint(geometry A, geometry B)''': Returns a float between 0 and 1 representing the location of the closest point on LineString to the given Point.
'''ST_Line_Substring(geometry A, double from, double to)''': Return a linestring being a substring of the input one starting and ending at the given fractions of total 2d length.
'''ST_Locate_Along_Measure(geometry A, double measure)''': Return a derived geometry collection value with elements that match the specified measure.
'''ST_Locate_Between_Measures(geometry A, double from, double to)''': Return a derived geometry collection value with elements that match the specified range of measures inclusively.
'''ST_AddMeasure(geometry A, double from, double to)''': Return a derived geometry with measure elements linearly interpolated between the start and end points. If the geometry has no measure dimension, one is added.

*3-D Functions:
'''ST_3DClosestPoint''' — Returns the 3-dimensional point on g1 that is closest to g2. This is the first point of the 3D shortest line.
'''ST_3DDistance''' — For geometry type Returns the 3-dimensional cartesian minimum distance (based on spatial ref) between two geometries in projected units.
'''ST_3DDWithin''' — For 3d (z) geometry type Returns true if two geometries 3d distance is within number of units.
'''ST_3DDFullyWithin''' — Returns true if all of the 3D geometries are within the specified distance of one another.
'''ST_3DIntersects''' — Returns TRUE if the Geometries “spatially intersect” in 3d - only for points and linestrings
'''ST_3DLongestLine''' — Returns the 3-dimensional longest line between two geometries
'''ST_3DMaxDistance''' — For geometry type Returns the 3-dimensional cartesian maximum distance (based on spatial ref) between two geometries in projected units.
'''ST_3DShortestLine''' — Returns the 3-dimensional shortest line between two geometries

*Relevant PostgreSQL Commands:
'''\dt *.*''' Show all tables
'''\q''' Exit table

*Specifities/ Outliers to consider:
New York (decompose)
Princeton area (keep Princeton unique)
Reston, Virginia (keep)
San Diego (include La Jolla)
Silicon Valley (all distinct)

* Continue reading from: https://postgis.net/docs/postgis_installation.html

2017-09-20:
* Attended first intro to GIS course yesterday
* Updated above notes on GIS

2017-09-19:
* Useful functions for spatial joins:

'''sum(expression)''': aggregate to return a sum for a set of records
'''count(expression)''': aggregate to return the size of a set of records
'''ST_Area(geometry)''' returns the area of the polygons
'''ST_AsText(geometry)''' returns WKT text
'''ST_Buffer(geometry, distance)''': For geometry: Returns a geometry that represents all points whose distance from this Geometry is less than or equal to distance. Calculations are in the Spatial Reference System of this Geometry. For geography: Uses a planar transform wrapper.
'''ST_Contains(geometry A, geometry B)''' returns the true if geometry A contains geometry B
'''ST_Distance(geometry A, geometry B)''' returns the minimum distance between geometry A and geometry B
'''ST_DWithin(geometry A, geometry B, radius)''' returns the true if geometry A is radius distance or less from geometry B
'''ST_GeomFromText(text)''' returns geometry
'''ST_Intersection(geometry A, geometry B)''': Returns a geometry that represents the shared portion of geomA and geomB. The geography implementation does a transform to geometry to do the intersection and then transform back to WGS84
'''ST_Intersects(geometry A, geometry B)''' returns the true if geometry A intersects geometry B
'''ST_Length(linestring)''' returns the length of the linestring
'''ST_Touches(geometry A, geometry B)''' returns the true if the boundary of geometry A touches geometry B
'''ST_Within(geometry A, geometry B)''' returns the true if geometry A is within geometry B
geometry_a '''&&''' geometry_b: Returns TRUE if A’s bounding box overlaps B’s.
geometry_a '''=''' geometry_b: Returns TRUE if A’s bounding box is the same as B’s.
'''ST_SetSRID(geometry, srid)''': Sets the SRID on a geometry to a particular integer value.
'''ST_SRID(geometry)''': Returns the spatial reference identifier for the ST_Geometry as defined in spatial_ref_sys table.
'''ST_Transform(geometry, srid)''': Returns a new geometry with its coordinates transformed to the SRID referenced by the integer parameter.
'''ST_Union()''': Returns a geometry that represents the point set union of the Geometries.
'''substring(string [from int] [for int])''': PostgreSQL string function to extract substring matching SQL regular expression.
'''ST_Relate(geometry A, geometry B)''': Returns a text string representing the DE9IM relationship between the geometries.
'''ST_GeoHash(geometry A)''': Returns a text string representing the GeoHash of the bounds of the object.

*Native functions for geogrphy:
'''ST_AsText(geography)''' returns text
'''ST_GeographyFromText(text)''' returns geography
'''ST_AsBinary(geography)''' returns bytea
'''ST_GeogFromWKB(bytea)''' returns geography
'''ST_AsSVG(geography)''' returns text
'''ST_AsGML(geography)''' returns text
'''ST_AsKML(geography)''' returns text
'''ST_AsGeoJson(geography)''' returns text
'''ST_Distance(geography, geography)''' returns double
'''ST_DWithin(geography, geography, float8)''' returns boolean
'''ST_Area(geography)''' returns double
'''ST_Length(geography)''' returns double
'''ST_Covers(geography, geography)''' returns boolean
'''ST_CoveredBy(geography, geography)''' returns boolean
'''ST_Intersects(geography, geography)''' returns boolean
'''ST_Buffer(geography, float8)''' returns geography [1]
'''ST_Intersection(geography, geography)''' returns geography [1]
* Continue reading from: http://workshops.boundlessgeo.com/postgis-intro/geography.html

2017-09-18:
* Read documentation on PostGIS and tiger geocoder
* Continue reading from: http://workshops.boundlessgeo.com/postgis-intro/joins.html

2017-09-12:
* Clarified University Matching output file.
* Helped Christy with pdf-reader, capturing keywords in readable format.

2017-09-11:
* Ensured that documentation exists for the projects worked on last semester.

</onlyinclude>

===Spring 2017===

2017-04-17:
* To pull accelerators: Wrote simple python regex-based script that ran on organizations data.
Code: E:\McNair\Projects\Accelerators\Crunchbase Snapshot\accelerator keywords.py
Matched output (885 mathces) : E:\McNair\Projects\Accelerators\Crunchbase Snapshot\Jeemin_885_accel_matches

2017-04-14:
* Loaded Federal Grants Data into database

2017-04-12:
* Finishing up cleaning the columns for Federal Grant Data - NIH. The output excel files can be accessed at:
E:\McNair\Projects\Federal Grant Data\NIH\Grants
Titled:
Jeemin_combined_files 1986-2001.csv
Jeemin_combined_files 2002-2012.csv
Jeemin_combined_files 2013-2015.csv

* psql table formula:

CREATE TABLE all_grants (
APPLICATION_ID integer,
ACTIVITY varchar(3),
ADMINISTERING_IC varchar(2),
APPLICATION_TYPE varchar(1),
ARRA_FUNDED varchar(1),
AWARD_NOTICE_DATE date,
BUDGET_START date,
BUDGET_END date,
CFDA_CODE varchar(3),
CORE_PROJECT_NUM varchar(11),
ED_INST_TYPE varchar(30),
FOA_NUMBER varchar(13),
FULL_PROJECT_NUM varchar(35),
FUNDING_ICs varchar(40),
FUNDING_MECHANISM varchar(23),
FY smallint,
IC_NAME varchar(77),
NIH_SPENDING_CATS varchar(295),
ORG_CITY varchar(20),
ORG_COUNTRY varchar(16),
ORG_DEPT varchar(30),
ORG_DISTRICT smallint,
ORG_DUNS integer,
ORG_FIPS varchar(2),
ORG_NAME varchar(60),
ORG_STATE varchar(2),
ORG_ZIPCODE integer,
PHR varchar(200),
PI_IDS varchar(30),
PI_NAMEs varchar(200),
PROGRAM_OFFICER_NAME varchar(36),
PROJECT_START date,
PROJECT_END date,
PROJECT_TERMS varchar(200),
PROJECT_TITLE varchar(244),
SERIAL_NUMBER smallint,
STUDY_SECTION varchar(4),
STUDY_SECTION_NAME varchar(100),
SUBPROJECT_ID smallint,
SUFFIX varchar(2),
SUPPORT_YEAR smallint,
DIRECT_COST_AMT integer,
INDIRECT_COST_AMT integer,
TOTAL_COST integer,
TOTAL_COST_SUB_PROJECT integer
);
\COPY all_grants FROM 'Jeemin_combined_files 1986-2001.csv' WITH DELIMITER AS E'\t' HEADER NULL AS ''CSV

2017-03-29:
* Troubled by the variety of cases - separating keys by keywords will not work favorably when it hits University of California vs. University of Southern California case - find a way to match University of Southern California first (more specific ones first) - but how to generalize

2017-03-27:
* Writing code for university matches - decided to go through keys instead of each dataitem. Use keywords in each key to go through the dataitem - misspellings are currently unaccounted for.

2017-03-24:
* Discussed with Julia & Meghana about university keys to use to count # of occurrences, including aliases and misspellings
* Thoughts: to use a scoring metric with a key of UNIVERSITY OF CALIFORNIA SYSTEM, it should have a 'better' score when compared to MATHEMATICAL SCIENCES PUBLISHERS C/O UNIVERSITY OF CALIFORNIA BERKELEY or CALIFORNIA AT LOS ANGELES, UNVIERSITY OF than when compared to UNIVERSITY OF SOUTHERN CALIFORNIA, which may pose a challenge when attempting to implement this in a more general sense. In normalizing a string, strip "THE", "," and split words by spaces and compare each keyword from the two strings. Deciding on which strings to compare will be another issue - length (within some range maybe) could be an option.
* Federal Grant Data XML Parser was rerun - same output textfiles

2017-03-22:
* Read string matching & calculating distance, below are relevant links
* [http://www.cs.cmu.edu/~wcohen/postscript/ijcai-ws-2003.pdf]
* [http://web.archive.org/web/20081224234350/http://www.dcs.shef.ac.uk/~sam/stringmetrics.html]

2017-03-22:
* Talked to Julia about universal matcher, want to combine all University of California's to University of California, The Regents of
* Converted crunchbase2013 data from mySQL to PostgreSQL, but having trouble with the last table - cb_relationships, complains about syntax error at or near some places - but generally all tables exist in database called crunchbase
* Federal Grant Data XML Parser was run - the three output textfiles can be found in E:\McNair\Projects\Federal Grant Data\NSF

2017-03-16:
* Further documented [[University Patent Matching]]
* Finished writing XML Parser

2017-03-15:
* Todo: write a wikipage on possible input/output info on string matcher
* Wrote part of XML parser, extracted yearly data into E:\McNair\Projects\Federal Grant Data\NSF\NSF Extracted Data (up to year 2010)

2017-03-14:
* Started pulling academy cases but there are too many cases to worry about, in terms of institution of interest. A document is located in E:\McNair\Projects\University Patents\academies_verify_cases.txt
* Need Julia/Meghana to look through the hits and see which are relevant & extract pattern from there.
* Having trouble outputting txt file without double quotes around every line.
* Thinking that one text file should be output for all keywords instead of having one each, to avoid overlap (ex) COLLEGE and UNIVERSITY are both keywords; ALBERT EINSTEIN COLLEGE OF YESHIVA UNIVERSITY will be hit twice if it were counted as two separate instances, one accounting for COLLEGE and the other for UNIVERSITY) - either in the form of if-elseif statements or one big regex check.

2017-03-13:
* For University Patent Data Matching - matched SCHOOL (output: E:\McNair\Projects\University Patents\school_pulled_from_assignee_list_USA) and matched INSTITUTE(output: E:\McNair\Projects\University Patents\institute_pulled_from_assignee_list_USA).
* [[University Patent Matching]]
* To be worked on later: Grant XML parsing & general name matcher

2017-03-08:
* Wrote regex pattern that identifies all "university" matchings - can be found in E:\McNair\Projects\University Patents\university_pulled_from_assignee_list_USA -- is an output file
* Talked to Sonia, but didn't come to solid conclusion on identifying whether key words associate with city or country by running a python function
* Output sql tables from finished run of Jeemin_FDATrial_as_key_data_ripping.py
* Ran through assigneelist_USA.txt to see how many different ways UNIVERSITY could be spelled wrong. There were many.
* Tried to logic through creating a pattern that could catch all different versions of UNIVERSITY. Discuss further on whether UNIVERSITIES and those that include UNIVERSITIES but include INC in the end should be pulled as relevant information

2017-03-06:
* [[Installing python in a database]]
* Added building Python function section to [[Working with PostgreSQL]] at the bottom of the page.
* Ran FDA Trial data ripping again, as the text output files were wiped.
* Plan on discussing with Julia and Meghana again about pulling universities and other relevant institutions from the Assignee List USA.
* Talked to Sonia about pulling city, state, zipcode information, hence python was installed in a database. Will work with Sonia on Wednesday afternoon and see how best a regex function could be implemented

2017-03-03:
* Attempted to output sql tables

2017-03-01:
* Started re-running Jeemin_FDATrial_as_key_data_ripping.py

2017-02-27:
* Finished producing tables from Jeemin_FDATrial_as_key_data_ripping.py
* Talked to Julia about LinkedIn data extracting - to be discussed further with Julia & Peter.
* Started web crawler for Wikipedia - currently pulls Endowment, Academic staff, students, undergraduates, and postgraduates info found on Rice Wikipedia page. Can be found in : E:\McNair\Projects\University Patents\Jeemin_University_wikipedia_crawler.py

2017-02-24:
* Continued working on producing multiple tables - first two are done. Was working on location, as there are multiple location tags per location.

2017-02-22:
* Finished Jeemin_FDATrial_as_key_data_ripping.py (E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_FDATrial_as_key_data_ripping.py), which outputs to E:\McNair\Projects\FDA Trials\Jeemin_Project\general_data_ripping_output.txt; TODO: output four different tables & replace the write in the same for-loop as going through each file

2017-02-20:
* Continued working on Jeemin_FDATrial_as_key_data_ripping.py to find tags and place all of those information in a list. The other zipcode file did not finish executing after 2+ hours of running it - considering the possibility of splitting the record file into smaller bits, or running the processing on a faster machine.

2017-02-17:
* Completed code for counting the number of occurrences for each unique zipcode. (currently titled & located: E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_Running_File.py). It has been running for 20+min because of the comprehensive XML data files. Meanwhile started coding to create a dictionary with the keys corresponding to each unique trial ID, mapped to every other information (location, sponsors, phase, drugs ...etc.) (currently titled & located: E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_FDATrial_as_key_data_ripping.py).

2017-02-15:
* Discussed with Catherine what to do with FDA Trial data and decided to have a dictionary with zip-codes as keys and number of trials occurred in that zipcode as values. Was still attempting to loop through the files without the code having to exist in the same directory as the XML files. Plan to write to excel via tsv, with zip-code as one column and # of occurrence as the other.

2017-02-13:
* Goals (for trials): 1) Build ER Diagram 2) For each entity, get XML snippet 3) Build a parser/ripper for single file; the python parser can be found at: E:\McNair\Projects\FDA Trials\Jeemin_Project

2017-02-08:
* Attempted to come up with possible cases for locating the description of accelerators - pick up from extracting bodies of text from the about page (given that it exists)
* [[Trial Data Project]]

2017-02-06:
* Set up wikiPage & remote desktop.
* Started working on python version of web crawler. So far it successfully prints out a catchphrase/ description for one website. To be worked on. The python file can be found in: E:\McNair\Projects\Accelerators\Python WebCrawler\webcrawlerpython.py

[[Category:Work Log]]

Jeemin Sim (Work Log)

2017-12-05T16:47:10Z

JeeminS:

[[Jeemin Sim]] [[Work Logs]] [[Jeemin Sim (WorkLog)|(log page)]]

===Fall 2017===
<onlyinclude>

2017-12-05:
* Uploaded tables with KML files in TIF folder in bulk(E:) drive
** Allegheny County
** Atlanta
** Chicago
** Columbus, OH
** Dublin, OH
** Vermont
** Washington, D.C.

*Updated documentation for:
http://mcnair.bakerinstitute.org/wiki/TIF_Project#Uploading_TIF_Data_onto_database_.28tigertest.29

2017-12-04:
* To upload KML file into database with specified table name:
researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
* chicagotif table now resides in tigertest database

2017-11-30:
* Displayed map with TIF districts and startups
* Location:
E:\McNair\Projects\Agglomeration\TIF\McNair_Color_Chicago_TIF_and_startups.png
* Added information and steps to ArcMap/ArcGIS documentation page
* Create a project page for 'Working with POSTGIS' and add instructions for uploading KML file onto POSTGIS
** Command used :
*** Logged in as : researcher@McNairDBServ
/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml
* chicago TIF kml file currently downloaded in tigertest with a table name of Layer0
** Figure out how to change layer name while loading kml file
** Instructions pulled from : http://wiki.wildsong.biz/index.php/Loading_data_into_PostGIS#Loading_data_from_KMZ_files

2017-11-28:
* Created and edited [[ArcMap / ArcGIS Documentation]]
* Plotted points for TIFS and Startups in Chicago in one map.
** Location:
E:\McNair\Projects\Agglomeration\TIF\Jeemin_Chicago_TIF_and_Startups_Attempt1
* Used https://mygeodata.cloud/result to convert from KML file to CSV (which was then saved as txt file to be uploaded onto ArcMap)
* Text files located in Local Disk (C:) Drive

2017-11-27:
*Download Chicago TIF data

2017-11-13:
*

2017-11-09:
* Notes on data downloaded:
** Year 2010-2012 data are based on total population, not 25 yrs or over (case for all other tables)
*** Record appears five times total, with same exact column name
***For exmaple: 'Total; Estimate; High school graduate (includes equivalency)' appears five times, with different values.
* TODO:
** Make Projects page for ACS Data
***[[American Community Survey (ACS) Data]]

2017-11-07:
*Yesterday, narrowed down columns of interest from ACS_S1501_educationattain_2016 table.

Id
Id2
Geography
Total; Estimate; Population 25 years and over
Total; Estimate; Population 25 years and over - High school graduate (includes equivalency)
Total; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Total; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Percent; Margin of Error; Population 25 years and over - High school graduate (includes equivalency)
Total; Estimate; Population 25 years and over - Associate's degree
Total; Margin of Error; Population 25 years and over - Associate's degree
Percent; Estimate; Population 25 years and over - Associate's degree
Percent; Margin of Error; Population 25 years and over - Associate's degree
Total; Estimate; Population 25 years and over - Bachelor's degree
Total; Margin of Error; Population 25 years and over - Bachelor's degree
Percent; Estimate; Population 25 years and over - Bachelor's degree
Percent; Margin of Error; Population 25 years and over - Bachelor's degree
Total; Estimate; Population 25 years and over - Graduate or professional degree
Total; Margin of Error; Population 25 years and over - Graduate or professional degree
Percent; Estimate; Population 25 years and over - Graduate or professional degree
Percent; Margin of Error; Population 25 years and over - Graduate or professional degree
Percent; Estimate; Percent high school graduate or higher
Percent; Margin of Error; Percent high school graduate or higher
Percent; Estimate; Percent bachelor's degree or higher
Percent; Margin of Error; Percent bachelor's degree or higher

*Complications:
**For csv files corresponding to years 2015 & 2016, all of the above columns exist.
**For csv files corresponding to years 2005 - 2014, no 'Percent' columns exist
*** Instead their 'Total' columns are percentage values
**For csv file corresponding to year 2005, columns regarding Graduate or professional degree are labeled differently.
**2012 data doesn't correspond to Population 25 years and over.

*Temporary Solution:
**Since the above problems may be specific to this set of tables, will go through csv files and adjust columns.

*Python script location:
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data\pullCertainColumns.py

2017-10-31:
* Finished doanloading files from ACS.
* Started loading tables into tigertest.
* Commands run could be found in
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data\DataLoading_SQL_Commands.txt

2017-10-30:
* Downloaded data from ACS, to be continued
* File path:
E:\McNair\Projects\Agglomeration\ACS_Downloaded_Data
* Fields of interest:
S1401 SCHOOL ENROLLMENT
S1501 EDUCATIONAL ATTAINMENT
S2301 EMPLOYMENT STATUS
B01003 TOTAL POPULATION
B02001 RACE
B07201 GEOGRAPHICAL MOBILITY
B08303 TRAVEL TIME TO WORK
B19013 MEDIAN HOUSEHOLD INCOME
B19053 SELF-EMPLOYMENT INCOME IN THE PAST 12 MONTHS FOR HOUSEHOLDS
B19083 GINI INDEX OF INCOME INEQUALITY
B25003 TENURE
B25105 MEDIAN MONTHLY HOUSING COSTS
B28011 INTERNET SUBSCRIPTIONS IN HOUSEHOLD
G001 GEOGRAPHIC IDENTIFIERS

2017-10-23:
* Talked to Ed with Peter & Oliver about upcoming tasks & projects.
* Loaded acs_place table 2017 (does not contain population) on tigertest.
** SQL commands used:

DROP TABLE acs_place;

CREATE TABLE acs_place (
USPS varchar(5),
GEOID varchar(30),
ANSICODE varchar(30),
NAME varchar(100),
LSAD varchar(30),
FUNCSTAT varchar(10),
ALAND varchar(30),
AWATER varchar(30),
ALAND_SQMI varchar(30),
AWATER_SQMI varchar(30),
INTPTLAT varchar(30),
INTPTLONG varchar(30)
);

\COPY acs_place FROM '/bulk/2017_Gaz_place_national.txt';
--COPY 29578

* TODO:
** Find acs place 2016 data for population
** Find larger acs files, ideally at the place level
** Provide more documentation on POSTGIS & geocoding

2017-10-16:
* Exported maps of points from the Bay Area each year.
** Map used location: E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS\Jeemin_Bay_Area Points_Every_Year\BayAreaEveryYearMap
** Zoom scale: 1:650.000
* Location of Bay Area Points png files:
E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS\Jeemin_Bay_Area Points_Every_Year

2017-10-10:
*Discoveries/ Struggles regarding.gdb to .shp file conversion:
** Esri Production Mapping (costly)
*** License needs to be purchasesd: http://www.esri.com/software/arcgis/extensions/production-mapping/pricing
** Use ogr2ogr from gdal package
*** https://gis.stackexchange.com/questions/14432/migrating-geodatabase-data-into-postgis-without-esri-apps
*** Command: ogr2ogr -f "ESRI Shapefile" [Destination of shapefile] [path to gdb file]
*** Problem installing gdal

2017-10-09:
* TODO'S:
** Downloading data onto tigertest
*** Road
*** Railway
*** Coastline
*** Instructions: http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation#Bulk_Download_TIGER_Shapefiles
** Configure census data from American Community Survey (ACS)
*** 1) Work out what data is of our interest (confirm ACS)
*** 2) Determine appropriate shape file unit:
**** census block vs. census block group vs. census track
*** 3) Load into tigertest

* Done:
** Downloaded data from https://www.census.gov/cgi-bin/geo/shapefiles/index.php
tl_2017_us_coastline -- 4209
tl_2017_us_primaryroads -- 11574
tl_2017_us_rails -- 176237
** Link found to potentially download ACS data: https://www.census.gov/geo/maps-data/data/tiger-data.html
*** But most files on it come with .gdb extension and not .shp

2017-10-03:
* Installed PostGIS & is now visible on pgAdmin III

* ArcGIS (connect to postgis database):
** 1) Open ArcMap
** 2) Either open blank or open existing file/project
** 3) Click on 'Add Data' button with a cross and a yellow diamond (under Selection toolbar)
** 4) Go to the top-most directory by pressing on the arrow that points left-then-up (on the left of home button)
** 5) Click on 'Database Connections'
** 6) Click on 'Add Database Connection' (if Connection to localhost.sde) does not exist already)
** 7) Fill in the following fields:
*** Database Platform: PostgreSQL
*** Instance: localhost
*** User name: postgres
*** Password:
*** Database: tigertest
** 8) Press 'OK'
** 9) Now you'll have 'Connection to localhost.sde' in your Database Connections
** 10) Double click on 'Connection to localhost.sde'
** 11) Double click on the table of interest
** 12) Click 'Finish'
** 13) You'll see information populated on map, as one of the 'Layers'
*** Tested with: tigertest.public.copointplacescontains

* On running & altering Oliver's script:
** Location: E:\McNair\Projects\OliverLovesCircles\src\python\vc_circles.py
** Ed manipulated file names so that underscores would replace dots (St.Louis --> St_Louis)
** Takes in instances and sweep times as part of the argument, but not impactful as those variables are hardcoded in the script
** Ran vc_circles.py with the following variables with changed values:
*** SWEEP_CYCLE_SECONDS = 10 (used to be 30)
*** NUMBER_INSTANCES = 16 (used to be 8)
** New output to be found in: E:\McNair\Projects\OliverLovesCircles\out

2017-10-02:
* Talked to Harrison & Peter regarding ArcGIS
** Currently have points plotted on Houston
** Trouble interpreting geometry type, as currently reads in from text file
** Documents located in : E:\McNair\Projects\Agglomeration\HarrisonPeterWorkArcGIS
* Attempted to install PostGIS spatial extention from PostgreSQL but getting 'spatial database creation failed' error message.
** Referenced instructions:
*** https://www.gpsfiledepot.com/tutorials/installing-and-setting-up-postgresql-with-postgis/
*** http://www.bostongis.com/PrinterFriendly.aspx?content_name=postgis_tut01

2017-09-26:
* Created a table that maps a state to the database name.
** http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation#Translating_Table_names_to_corresponding_States
* Added more GIS-information (functions, realm & outliers to consider)
** http://mcnair.bakerinstitute.org/wiki/Urban_Start-up_Agglomeration#GIS_Resources
* Visualization in PostGIS or connecting to ArcGIS for visualization (import/export data)
* Spatial indexing:
** http://revenant.ca/www/postgis/workshop/indexing.html

2017-09-25:
* Talked to Ed about GIS, Census data, and going about determining the correctness of reported 'place.' Currently script makes a cross product of each reported place and an existing place, outputting a column of boolean value to indicate whether the reported place's coordinates fell within a place's geometric boundaries. One other way of going about this which we discussed is to first check if the reported place does fall within that place's boundaries. If it isn't, we'll go about the cross product method.

* To add documentation :
** http://mcnair.bakerinstitute.org/wiki/PostGIS_Installation
** http://mcnair.bakerinstitute.org/wiki/Urban_Start-up_Agglomeration

* Discussed the need to maintain venture capital database.

*Relevant File paths:
**E:\McNair\Projects\Agglomeration\TestGIS.sql
**Z:\VentureCapitalData\SDCVCData\vcdb2\ProecssingCoLevelSimple.sql
**Z:\VentureCapitalData\SDCVCData\vcdb2\CitiesWithGT10Active.txt

2017-09-21:
* Functions for Linear Referencing:
'''ST_LineInterpolatePoint(geometry A, double measure)''': Returns a point interpolated along a line.
'''ST_LineLocatePoint(geometry A, geometry B)''': Returns a float between 0 and 1 representing the location of the closest point on LineString to the given Point.
'''ST_Line_Substring(geometry A, double from, double to)''': Return a linestring being a substring of the input one starting and ending at the given fractions of total 2d length.
'''ST_Locate_Along_Measure(geometry A, double measure)''': Return a derived geometry collection value with elements that match the specified measure.
'''ST_Locate_Between_Measures(geometry A, double from, double to)''': Return a derived geometry collection value with elements that match the specified range of measures inclusively.
'''ST_AddMeasure(geometry A, double from, double to)''': Return a derived geometry with measure elements linearly interpolated between the start and end points. If the geometry has no measure dimension, one is added.

*3-D Functions:
'''ST_3DClosestPoint''' — Returns the 3-dimensional point on g1 that is closest to g2. This is the first point of the 3D shortest line.
'''ST_3DDistance''' — For geometry type Returns the 3-dimensional cartesian minimum distance (based on spatial ref) between two geometries in projected units.
'''ST_3DDWithin''' — For 3d (z) geometry type Returns true if two geometries 3d distance is within number of units.
'''ST_3DDFullyWithin''' — Returns true if all of the 3D geometries are within the specified distance of one another.
'''ST_3DIntersects''' — Returns TRUE if the Geometries “spatially intersect” in 3d - only for points and linestrings
'''ST_3DLongestLine''' — Returns the 3-dimensional longest line between two geometries
'''ST_3DMaxDistance''' — For geometry type Returns the 3-dimensional cartesian maximum distance (based on spatial ref) between two geometries in projected units.
'''ST_3DShortestLine''' — Returns the 3-dimensional shortest line between two geometries

*Relevant PostgreSQL Commands:
'''\dt *.*''' Show all tables
'''\q''' Exit table

*Specifities/ Outliers to consider:
New York (decompose)
Princeton area (keep Princeton unique)
Reston, Virginia (keep)
San Diego (include La Jolla)
Silicon Valley (all distinct)

* Continue reading from: https://postgis.net/docs/postgis_installation.html

2017-09-20:
* Attended first intro to GIS course yesterday
* Updated above notes on GIS

2017-09-19:
* Useful functions for spatial joins:

'''sum(expression)''': aggregate to return a sum for a set of records
'''count(expression)''': aggregate to return the size of a set of records
'''ST_Area(geometry)''' returns the area of the polygons
'''ST_AsText(geometry)''' returns WKT text
'''ST_Buffer(geometry, distance)''': For geometry: Returns a geometry that represents all points whose distance from this Geometry is less than or equal to distance. Calculations are in the Spatial Reference System of this Geometry. For geography: Uses a planar transform wrapper.
'''ST_Contains(geometry A, geometry B)''' returns the true if geometry A contains geometry B
'''ST_Distance(geometry A, geometry B)''' returns the minimum distance between geometry A and geometry B
'''ST_DWithin(geometry A, geometry B, radius)''' returns the true if geometry A is radius distance or less from geometry B
'''ST_GeomFromText(text)''' returns geometry
'''ST_Intersection(geometry A, geometry B)''': Returns a geometry that represents the shared portion of geomA and geomB. The geography implementation does a transform to geometry to do the intersection and then transform back to WGS84
'''ST_Intersects(geometry A, geometry B)''' returns the true if geometry A intersects geometry B
'''ST_Length(linestring)''' returns the length of the linestring
'''ST_Touches(geometry A, geometry B)''' returns the true if the boundary of geometry A touches geometry B
'''ST_Within(geometry A, geometry B)''' returns the true if geometry A is within geometry B
geometry_a '''&&''' geometry_b: Returns TRUE if A’s bounding box overlaps B’s.
geometry_a '''=''' geometry_b: Returns TRUE if A’s bounding box is the same as B’s.
'''ST_SetSRID(geometry, srid)''': Sets the SRID on a geometry to a particular integer value.
'''ST_SRID(geometry)''': Returns the spatial reference identifier for the ST_Geometry as defined in spatial_ref_sys table.
'''ST_Transform(geometry, srid)''': Returns a new geometry with its coordinates transformed to the SRID referenced by the integer parameter.
'''ST_Union()''': Returns a geometry that represents the point set union of the Geometries.
'''substring(string [from int] [for int])''': PostgreSQL string function to extract substring matching SQL regular expression.
'''ST_Relate(geometry A, geometry B)''': Returns a text string representing the DE9IM relationship between the geometries.
'''ST_GeoHash(geometry A)''': Returns a text string representing the GeoHash of the bounds of the object.

*Native functions for geogrphy:
'''ST_AsText(geography)''' returns text
'''ST_GeographyFromText(text)''' returns geography
'''ST_AsBinary(geography)''' returns bytea
'''ST_GeogFromWKB(bytea)''' returns geography
'''ST_AsSVG(geography)''' returns text
'''ST_AsGML(geography)''' returns text
'''ST_AsKML(geography)''' returns text
'''ST_AsGeoJson(geography)''' returns text
'''ST_Distance(geography, geography)''' returns double
'''ST_DWithin(geography, geography, float8)''' returns boolean
'''ST_Area(geography)''' returns double
'''ST_Length(geography)''' returns double
'''ST_Covers(geography, geography)''' returns boolean
'''ST_CoveredBy(geography, geography)''' returns boolean
'''ST_Intersects(geography, geography)''' returns boolean
'''ST_Buffer(geography, float8)''' returns geography [1]
'''ST_Intersection(geography, geography)''' returns geography [1]
* Continue reading from: http://workshops.boundlessgeo.com/postgis-intro/geography.html

2017-09-18:
* Read documentation on PostGIS and tiger geocoder
* Continue reading from: http://workshops.boundlessgeo.com/postgis-intro/joins.html

2017-09-12:
* Clarified University Matching output file.
* Helped Christy with pdf-reader, capturing keywords in readable format.

2017-09-11:
* Ensured that documentation exists for the projects worked on last semester.

</onlyinclude>

===Spring 2017===

2017-04-17:
* To pull accelerators: Wrote simple python regex-based script that ran on organizations data.
Code: E:\McNair\Projects\Accelerators\Crunchbase Snapshot\accelerator keywords.py
Matched output (885 mathces) : E:\McNair\Projects\Accelerators\Crunchbase Snapshot\Jeemin_885_accel_matches

2017-04-14:
* Loaded Federal Grants Data into database

2017-04-12:
* Finishing up cleaning the columns for Federal Grant Data - NIH. The output excel files can be accessed at:
E:\McNair\Projects\Federal Grant Data\NIH\Grants
Titled:
Jeemin_combined_files 1986-2001.csv
Jeemin_combined_files 2002-2012.csv
Jeemin_combined_files 2013-2015.csv

* psql table formula:

CREATE TABLE all_grants (
APPLICATION_ID integer,
ACTIVITY varchar(3),
ADMINISTERING_IC varchar(2),
APPLICATION_TYPE varchar(1),
ARRA_FUNDED varchar(1),
AWARD_NOTICE_DATE date,
BUDGET_START date,
BUDGET_END date,
CFDA_CODE varchar(3),
CORE_PROJECT_NUM varchar(11),
ED_INST_TYPE varchar(30),
FOA_NUMBER varchar(13),
FULL_PROJECT_NUM varchar(35),
FUNDING_ICs varchar(40),
FUNDING_MECHANISM varchar(23),
FY smallint,
IC_NAME varchar(77),
NIH_SPENDING_CATS varchar(295),
ORG_CITY varchar(20),
ORG_COUNTRY varchar(16),
ORG_DEPT varchar(30),
ORG_DISTRICT smallint,
ORG_DUNS integer,
ORG_FIPS varchar(2),
ORG_NAME varchar(60),
ORG_STATE varchar(2),
ORG_ZIPCODE integer,
PHR varchar(200),
PI_IDS varchar(30),
PI_NAMEs varchar(200),
PROGRAM_OFFICER_NAME varchar(36),
PROJECT_START date,
PROJECT_END date,
PROJECT_TERMS varchar(200),
PROJECT_TITLE varchar(244),
SERIAL_NUMBER smallint,
STUDY_SECTION varchar(4),
STUDY_SECTION_NAME varchar(100),
SUBPROJECT_ID smallint,
SUFFIX varchar(2),
SUPPORT_YEAR smallint,
DIRECT_COST_AMT integer,
INDIRECT_COST_AMT integer,
TOTAL_COST integer,
TOTAL_COST_SUB_PROJECT integer
);
\COPY all_grants FROM 'Jeemin_combined_files 1986-2001.csv' WITH DELIMITER AS E'\t' HEADER NULL AS ''CSV

2017-03-29:
* Troubled by the variety of cases - separating keys by keywords will not work favorably when it hits University of California vs. University of Southern California case - find a way to match University of Southern California first (more specific ones first) - but how to generalize

2017-03-27:
* Writing code for university matches - decided to go through keys instead of each dataitem. Use keywords in each key to go through the dataitem - misspellings are currently unaccounted for.

2017-03-24:
* Discussed with Julia & Meghana about university keys to use to count # of occurrences, including aliases and misspellings
* Thoughts: to use a scoring metric with a key of UNIVERSITY OF CALIFORNIA SYSTEM, it should have a 'better' score when compared to MATHEMATICAL SCIENCES PUBLISHERS C/O UNIVERSITY OF CALIFORNIA BERKELEY or CALIFORNIA AT LOS ANGELES, UNVIERSITY OF than when compared to UNIVERSITY OF SOUTHERN CALIFORNIA, which may pose a challenge when attempting to implement this in a more general sense. In normalizing a string, strip "THE", "," and split words by spaces and compare each keyword from the two strings. Deciding on which strings to compare will be another issue - length (within some range maybe) could be an option.
* Federal Grant Data XML Parser was rerun - same output textfiles

2017-03-22:
* Read string matching & calculating distance, below are relevant links
* [http://www.cs.cmu.edu/~wcohen/postscript/ijcai-ws-2003.pdf]
* [http://web.archive.org/web/20081224234350/http://www.dcs.shef.ac.uk/~sam/stringmetrics.html]

2017-03-22:
* Talked to Julia about universal matcher, want to combine all University of California's to University of California, The Regents of
* Converted crunchbase2013 data from mySQL to PostgreSQL, but having trouble with the last table - cb_relationships, complains about syntax error at or near some places - but generally all tables exist in database called crunchbase
* Federal Grant Data XML Parser was run - the three output textfiles can be found in E:\McNair\Projects\Federal Grant Data\NSF

2017-03-16:
* Further documented [[University Patent Matching]]
* Finished writing XML Parser

2017-03-15:
* Todo: write a wikipage on possible input/output info on string matcher
* Wrote part of XML parser, extracted yearly data into E:\McNair\Projects\Federal Grant Data\NSF\NSF Extracted Data (up to year 2010)

2017-03-14:
* Started pulling academy cases but there are too many cases to worry about, in terms of institution of interest. A document is located in E:\McNair\Projects\University Patents\academies_verify_cases.txt
* Need Julia/Meghana to look through the hits and see which are relevant & extract pattern from there.
* Having trouble outputting txt file without double quotes around every line.
* Thinking that one text file should be output for all keywords instead of having one each, to avoid overlap (ex) COLLEGE and UNIVERSITY are both keywords; ALBERT EINSTEIN COLLEGE OF YESHIVA UNIVERSITY will be hit twice if it were counted as two separate instances, one accounting for COLLEGE and the other for UNIVERSITY) - either in the form of if-elseif statements or one big regex check.

2017-03-13:
* For University Patent Data Matching - matched SCHOOL (output: E:\McNair\Projects\University Patents\school_pulled_from_assignee_list_USA) and matched INSTITUTE(output: E:\McNair\Projects\University Patents\institute_pulled_from_assignee_list_USA).
* [[University Patent Matching]]
* To be worked on later: Grant XML parsing & general name matcher

2017-03-08:
* Wrote regex pattern that identifies all "university" matchings - can be found in E:\McNair\Projects\University Patents\university_pulled_from_assignee_list_USA -- is an output file
* Talked to Sonia, but didn't come to solid conclusion on identifying whether key words associate with city or country by running a python function
* Output sql tables from finished run of Jeemin_FDATrial_as_key_data_ripping.py
* Ran through assigneelist_USA.txt to see how many different ways UNIVERSITY could be spelled wrong. There were many.
* Tried to logic through creating a pattern that could catch all different versions of UNIVERSITY. Discuss further on whether UNIVERSITIES and those that include UNIVERSITIES but include INC in the end should be pulled as relevant information

2017-03-06:
* [[Installing python in a database]]
* Added building Python function section to [[Working with PostgreSQL]] at the bottom of the page.
* Ran FDA Trial data ripping again, as the text output files were wiped.
* Plan on discussing with Julia and Meghana again about pulling universities and other relevant institutions from the Assignee List USA.
* Talked to Sonia about pulling city, state, zipcode information, hence python was installed in a database. Will work with Sonia on Wednesday afternoon and see how best a regex function could be implemented

2017-03-03:
* Attempted to output sql tables

2017-03-01:
* Started re-running Jeemin_FDATrial_as_key_data_ripping.py

2017-02-27:
* Finished producing tables from Jeemin_FDATrial_as_key_data_ripping.py
* Talked to Julia about LinkedIn data extracting - to be discussed further with Julia & Peter.
* Started web crawler for Wikipedia - currently pulls Endowment, Academic staff, students, undergraduates, and postgraduates info found on Rice Wikipedia page. Can be found in : E:\McNair\Projects\University Patents\Jeemin_University_wikipedia_crawler.py

2017-02-24:
* Continued working on producing multiple tables - first two are done. Was working on location, as there are multiple location tags per location.

2017-02-22:
* Finished Jeemin_FDATrial_as_key_data_ripping.py (E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_FDATrial_as_key_data_ripping.py), which outputs to E:\McNair\Projects\FDA Trials\Jeemin_Project\general_data_ripping_output.txt; TODO: output four different tables & replace the write in the same for-loop as going through each file

2017-02-20:
* Continued working on Jeemin_FDATrial_as_key_data_ripping.py to find tags and place all of those information in a list. The other zipcode file did not finish executing after 2+ hours of running it - considering the possibility of splitting the record file into smaller bits, or running the processing on a faster machine.

2017-02-17:
* Completed code for counting the number of occurrences for each unique zipcode. (currently titled & located: E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_Running_File.py). It has been running for 20+min because of the comprehensive XML data files. Meanwhile started coding to create a dictionary with the keys corresponding to each unique trial ID, mapped to every other information (location, sponsors, phase, drugs ...etc.) (currently titled & located: E:\McNair\Projects\FDA Trials\Jeemin_Project\Jeemin_FDATrial_as_key_data_ripping.py).

2017-02-15:
* Discussed with Catherine what to do with FDA Trial data and decided to have a dictionary with zip-codes as keys and number of trials occurred in that zipcode as values. Was still attempting to loop through the files without the code having to exist in the same directory as the XML files. Plan to write to excel via tsv, with zip-code as one column and # of occurrence as the other.

2017-02-13:
* Goals (for trials): 1) Build ER Diagram 2) For each entity, get XML snippet 3) Build a parser/ripper for single file; the python parser can be found at: E:\McNair\Projects\FDA Trials\Jeemin_Project

2017-02-08:
* Attempted to come up with possible cases for locating the description of accelerators - pick up from extracting bodies of text from the about page (given that it exists)
* [[Trial Data Project]]

2017-02-06:
* Set up wikiPage & remote desktop.
* Started working on python version of web crawler. So far it successfully prints out a catchphrase/ description for one website. To be worked on. The python file can be found in: E:\McNair\Projects\Accelerators\Python WebCrawler\webcrawlerpython.py

[[Category:Work Log]]

TIF Project

2017-12-05T16:44:06Z

JeeminS: /* Vermont */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== To view all columns of a tale ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Vermont ===

Table name: vermonttif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('vermonttif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid_1 | integer | | | | plain | |
status | character varying | | | | extended | |
tif | character varying | | | | extended | |
acresotv | double precision | | | | plain | |
parcels | integer | | | | plain | |
edtaxablev | integer | | | | plain | |
munitaxabl | integer | | | | plain | |
contactnam | character varying | | | | extended | |
contacttit | character varying | | | | extended | |
contacttel | character varying | | | | extended | |
contactema | character varying | | | | extended | |
stateconta | character varying | | | | extended | |
statetitle | character varying | | | | extended | |
statetelep | character varying | | | | extended | |
stateemail | character varying | | | | extended | |
datelastup | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

TIF Project

2017-12-05T16:43:27Z

JeeminS: /* Dublin, OH */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== To view all columns of a tale ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
--------------+--------------------------+-----------+----------+--------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dublintif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
tif_id | integer | | | | plain | |
tif_name | character varying | | | | extended | |
tif_type | character varying | | | | extended | |
ord_date | character varying | | | | extended | |
ord_num | character varying | | | | extended | |
ord_mod1 | character varying | | | | extended | |
ord_mod2 | character varying | | | | extended | |
ord_mod3 | character varying | | | | extended | |
pr_imp_st | character varying | | | | extended | |
pr_imp_val | integer | | | | plain | |
pub_imp_st | character varying | | | | extended | |
pub_imp_val | integer | | | | plain | |
num_jobs | integer | | | | plain | |
exp_date | integer | | | | plain | |
globalid | character varying | | | | extended | |
shapestarea | double precision | | | | plain | |
shapestlength | double precision | | | | plain | |

=== Vermont ===

Table name: vermonttif

TIF Project

2017-12-05T16:42:33Z

JeeminS: /* Uploading TIF Data onto database (tigertest) */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== To view all columns of a tale ==

\d+ <tablename>

== Cities & Tables ==

=== Chicago ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

=== Vermont ===

Table name: vermonttif

TIF Project

2017-12-05T16:41:52Z

JeeminS: /* Washington, D.C. */

{{McNair Projects
|Has title=TIF Project
|Has owner=Cindy Ryoo, Yunnie Huang,
|Has start date=11/20/2017
|Has deadline=12/09/2017
|Has keywords=TIF; city; data; GIS; Tax Increment Finance; KML
|Has project status=Active
}}
=TIF Data=

Data is in
E:\McNair\Projects\Agglomeration\TIF

==Cities in our sample that have GIS data===

===Chicago,IL===
Chicago, IL is in our data with 179 max active.

2017 GIS Data it appears to contain everything that we might need (the CSV contains everything)!!!
https://data.cityofchicago.org/Community-Economic-Development/Boundaries-Tax-Increment-Financing-Districts/fz5x-7zak

Other
Data on all tifs (inc start and end date) up to 2012
https://data.cityofchicago.org/Community-Economic-Development/TIF-Status-and-Eligibility/3qsz-jemf

Map (but not GIS) based data of Chicago TIFs up to 2017. Included CDC date.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects-Map/v3a3-hhqn

This dataset is a comprehensive list of every project in every TIF District that has received funding from the City of Chicago via the TIF program from the inception of TIF to current.
https://data.cityofchicago.org/Community-Economic-Development/Tax-Increment-Financing-TIF-Projects/mex4-ppfc

Chicago KML from another source
https://catalog.data.gov/dataset/boundaries-tax-increment-financing-districts

===Columbus, OH===
Columbus,OH is in our data with 29 max active
KML with start dates!
http://catalog.smartcolumbuside.com/en/dataset/tax-increment-financing-areas1

===Washington, DC===
Washington, DC is in our data with 63 max active
Data from 2011 available with TIF dates:
http://opendata.dc.gov/datasets/tax-increment-financing-tif-areas
See also, high tech dev zones (2013):
http://opendata.dc.gov/datasets/111b31a212814aad9320d301ccf9d6a9_20
Econ dev zones (2004)
http://opendata.dc.gov/datasets/a3aefd57db394fd68d739556253dc44d_18

===Houston, TX===
Houston, TX is in our data with 101 max active
Last updated 2017, appears to have dates!
http://data.houstontx.gov/dataset/city-tax-increment-reinvestment-zones-tirz

===Dallas, TX===

Has a map: http://www.arcgis.com/home/webmap/viewer.html?url=https://gis.dallascityhall.com/wwwgis/rest/services/Eco_public/EconomicDevelopment/MapServer/1
But the data doesn't contain dates and doesn't appear to be exportable.
It looks like there is Dallas data with dates from 2016: http://www.dallascad.org/GISDataProducts.aspx

==Vermont==
Burlington, VT is in our data with 10 max active

GIS KML file, but doesn't contain creation date
http://geodata.vermont.gov/datasets/cd53b08d40fc4e7dbc2757c36d038af5_7/data

Think the same thing is here:
https://catalog.data.gov/dataset/vt-tax-increment-financing-tif-districts

Overview info:
http://accd.vermont.gov/community-development/funding-incentives/tif

===Allegheny County, PA===

PA cities in our data:
Bethlehem PA 10 Lehigh and Northampton counties
Chesterbrook PA 11 Chester County
Horsham PA 10 Montgomery County
King of Prussia PA 15 Montgomery County
Philadelphia PA 123 Philadelphia County
Pittsburgh PA 150 Allegheny County
West Conshohocken PA Montgomery County

Pittsburgh,PA is in Allegheny County, and may be covered in the data
KML that is supposed to have dates but doesn't appear to
https://data.wprdc.org/dataset/allegheny-county-tif-boundaries

==Cities not in our sample==

===Bloomington, IN===
Bloomington, IN is not in our data

Some maps that plot TIFs, suggesting underlying data...
https://data.bloomington.in.gov/en/dataset/tax-increment-financing-tif-district-maps

They don't host the GIS data on their own site:
https://data.bloomington.in.gov/en/dataset?sort=score+desc%2C+metadata_modified+desc&res_format=KML&q=&page=2

===Dublin, OH===
Dublin, OH is not in our data
KML GIS data, has creation and expiration date!
http://data-dublinohio.opendata.arcgis.com/datasets/ff0af920c18e4aa78459137b6efcd3d3_0?uiTab=table&selectedAttributes%5B%5D=TIF_ID&chartType=bar

===Oakland, MI===
Oakland, MI is not in our data
KML that has TIFs without dates
http://accessoakland.oakgov.com/datasets/e90026448daa4d568daa3ef727d8d758_2

==Cities We Couldn't Find==
*Denver, CO
*Bellevue, WA
*Nashville-Davidson metropolitan government (balance), TN
*Washington, DC
*South San Francisco, CA
*Portland, OR
*Durham, NC
*Pleasanton, CA
*Baltimore, MD
*Burlington, MA
*Carlsbad, CA
*Milpitas, CA
*Richardson, TX
*Lexington, MA
*Los Gatos, CA
*Memphis, TN
*Oakland, CA
*Charlotte, NC
*Reston, VA
*Tysons Corner, VA
*Sandy Springs, GA
*Campbell, CA
*Madison, WI
*St. Louis, MO
*Alpharetta, GA
*Berkeley, CA
*Raleigh, NC
*Phoenix, AZ
*Ann Arbor, MI
*Woburn, MA
*Cary, NC
*Emeryville, CA
*Marlborough, MA
*Burlingame, CA
*Salt Lake City, UT
*San Carlos, CA
*Plano, TX
*Scottsdale, AZ
*Stamford, CT
*Alexandria, VA
*Newton, MA
*Pasadena, CA
*Albuquerque, NM
*Arlington, VA
*Bethesda, MD
*Columbus, OH
*Florence-Graham, CA
*Kirkland, WA
*Los Altos, CA
*Cleveland, OH
*Minneapolis, MN
*Boca Raton, FL
*Gaithersburg, MD
*New Haven, CT
*Cincinnati, OH
*Eden Prairie, MN
*Foster City, CA
*Indianapolis city (balance), IN
*Peachtree Corners, GA
*Columbia, MD
*Hayward, CA
*Irving, TX
*Bothell, WA
*Watertown Town, MA
*Chantilly, VA
*Culver City, CA
*Norwalk, CT
*El Segundo, CA
*Herndon, VA
*Providence, RI

There's data on a couple of places that we don't care about on https://ckan.geoplatform.gov/#?q=tax%20increment

==Other Stuff==

===St. Louis, MI===

St. Louis has data but without GIS shapes. The data is here: http://www.bettertogetherstl.com/studies/economic-development/tif-database . It was apparently extracted from a 2015 report here: http://dor.mo.gov/pdf/2015TIFAnnualReport.pdf . The locations are google mapped here: http://www.bettertogetherstl.com/tax-incremental-financing-map

===Philly===

City of Philly Enterprise Zones:

Enterprise Zones - Data includes commercial and industrial zones, i.e. areas with specific federal enterprise zone designation meant to attract and support businesses in blighted areas. Blighted areas are defined as meeting one of seven city mandated criteria, including unsafe, unsanitary and inadequate conditions; economically or socially undesirable land use; and faulty street and lot layout. https://www.opendataphilly.org/dataset/enterprise-zones

Also Empowerment Zones. https://www.opendataphilly.org/dataset/empowerment-zones

===Larimer County===

Larimer County Colorado: https://www.larimer.org/assessor/tif (no data)

===New Hampshire===

New Hampshire report (2003) says that "some have GIS data" http://www.swrpc.org/files/data/com_econ_dev/ed/December%202003_%20TIF%20Handbook_7232010forweb.pdf
We have Nashua, NH (max 14 active) in our data, but it isn't one that has GIS data.

===Sherman, TX===

Sherman, TX has GIS data without dates: https://koordinates.com/layer/16403-finance-tax-increment-finance-tif/

===Cincinnati===

Cincinnati has TIFs (http://choosecincy.com/Economic-Development/Programs-Services/Incentives-Financing/Tax-Increment-Financing.aspx) but no GIS data...

===Michigan===

Michigan as some state level data (not GIS) in summary somewhere because it is reference in this report: http://www.senate.michigan.gov/SFA/Publications/Notes/2016Notes/NotesWin16dk.pdf

==Academic/Policy papers==

A policy paper on the econ effects of TIFs https://projects.cberdata.org/reports/TifEconEffects-012815.pdf

=Uploading TIF Data onto database (tigertest)=

== Command to upload KML file into a table in database (tigertest) ==

researcher@McNairDBServ:/bulk/tigertest$ ogr2ogr -f PostgreSQL PG:"dbname=tigertest" chicagotif.kml -nln chicagotif
(chicagotif.kml is the name of the KML file and chicagotif is the name I wish to name the table)

== Cities & Tables ==

=== Chicago ===

Table name: chicagotif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('chicagotif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
name_trim | character varying | | | | extended | |
show | character varying | | | | extended | |
approval_d | character varying | | | | extended | |
ref | character varying | | | | extended | |
objectid | character varying | | | | extended | |
ind | character varying | | | | extended | |
repealed_d | character varying | | | | extended | |
comm_area | character varying | | | | extended | |
wards | character varying | | | | extended | |
shape_area | character varying | | | | extended | |
sbif | character varying | | | | extended | |
shape_len | character varying | | | | extended | |
type | character varying | | | | extended | |
expiration | character varying | | | | extended | |
use | character varying | | | | extended | |

=== Allegheny County, PA ===

Table name: alleghenycountytif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------------+--------------------------+-----------+----------+-----------------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('alleghenycountytif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
terminationdate | character varying | | | | extended | |
effectivedate | character varying | | | | extended | |
sponsor | character varying | | | | extended | |
active | character varying | | | | extended | |
verified | character varying | | | | extended | |
notes | character varying | | | | extended | |
verified_date | character varying | | | | extended | |

=== Atlanta ===

Table name: atlantatif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+---------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('atlantatif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
id | integer | | | | plain | |
area | integer | | | | plain | |
tad_name | character varying | | | | extended | |
start_ | character varying | | | | extended | |
status | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Columbus, OH ===

Table name: columbustif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------------+--------------------------+-----------+----------+----------------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('columbustif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
fund_no | character varying | | | | extended | |
orc | character varying | | | | extended | |
ordinance | character varying | | | | extended | |
passage | character varying | | | | extended | |
agreement | character varying | | | | extended | |
dte_status | character varying | | | | extended | |
acres | double precision | | | | plain | |
url | character varying | | | | extended | |
last_edit_date | character varying | | | | extended | |

=== Washington, D.C. ===

Table name: dctif

Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-------------+--------------------------+-----------+----------+----------------------------------------+----------+--------------+-------------
ogc_fid | integer | | not null | nextval('dctif_ogc_fid_seq'::regclass) | plain | |
wkb_geometry | geometry(Geometry,4326) | | | | main | |
name | character varying | | | | extended | |
description | character varying | | | | extended | |
timestamp | timestamp with time zone | | | | plain | |
begin | timestamp with time zone | | | | plain | |
end | timestamp with time zone | | | | plain | |
altitudemode | character varying | | | | extended | |
tessellate | integer | | | | plain | |
extrude | integer | | | | plain | |
visibility | integer | | | | plain | |
draworder | integer | | | | plain | |
icon | character varying | | | | extended | |
objectid | integer | | | | plain | |
url | character varying | | | | extended | |
dc_code | character varying | | | | extended | |
type | character varying | | | | extended | |
gis_id | character varying | | | | extended | |
shape_length | double precision | | | | plain | |
shape_area | double precision | | | | plain | |

=== Dublin, OH ===

Table name: dublintif

=== Vermont ===

Table name: vermonttif

TIF Project

2017-12-05T16:41:11Z

JeeminS: /* Columbus, OH */

TIF Project

2017-12-05T16:40:31Z

JeeminS: /* Atlanta */

TIF Project

2017-12-05T16:39:59Z

JeeminS: /* Uploading KML file into a table in database (tigertest) */

TIF Project

2017-12-05T16:39:37Z

JeeminS: /* Uploading KML file */

TIF Project

2017-12-05T16:39:12Z

JeeminS: /* Allegheny County, PA */

TIF Project

2017-12-05T16:38:42Z

JeeminS: /* Allegheny County, PA */

TIF Project

2017-12-05T16:37:10Z

JeeminS: /* Cities & Tables */

TIF Project

2017-12-05T16:33:09Z

JeeminS: /* Chicago */

TIF Project

2017-12-05T16:29:51Z

JeeminS: /* Chicago */