OKC Project - Revision history

imported>Ed at 20:00, 19 December 2011

2011-12-19T20:00:50Z

imported>Ed at 19:59, 19 December 2011

2011-12-19T19:59:27Z

imported>Ed at 19:57, 19 December 2011

2011-12-19T19:57:59Z

imported>Ed at 19:57, 19 December 2011

2011-12-19T19:57:27Z

imported>Ed at 19:56, 19 December 2011

2011-12-19T19:56:26Z

imported>Ed at 19:56, 19 December 2011

2011-12-19T19:56:10Z

imported>Ed at 19:55, 19 December 2011

2011-12-19T19:55:30Z

imported>Rani at 21:28, 8 December 2011

2011-12-08T21:28:54Z

imported>Ed at 04:19, 30 November 2011

2011-11-30T04:19:38Z

imported>Ed at 21:01, 26 September 2011

2011-09-26T21:01:35Z

@@ Line 13: / Line 13: @@
 ==Setup Instructions for Usha==
 #Go to vpn.hbs.edu and click Network Connect -> Start
 #Using SSH client, connect to researchgrid.hbs.edu

← Older revision		Revision as of 19:59, 19 December 2011
Line 9:		Line 9:
	*[[:Image:OKC2.sql.asci\|OKC2.sql]]		*[[:Image:OKC2.sql.asci\|OKC2.sql]]

		+	These files are self-documenting, and the second file builds out the current base table set containing: '''users''', '''views''', '''messages''', and '''delta'''.

	==Setup Instructions for Usha==		==Setup Instructions for Usha==

@@ Line 6: / Line 6: @@
 There are two SQL script files that were written to process the data:
-*[[:File:OKC1.sql.asci|OKC1.sql]]
+*[[:Image:OKC1.sql.asci|OKC1.sql]]
-*[[:File:OKC2.sql.asci|OKC2.sql]]
+*[[:Image:OKC2.sql.asci|OKC2.sql]]

@@ Line 6: / Line 6: @@
 There are two SQL script files that were written to process the data:
-*[[Image:OKC1.sql.asci OKC1.sql]]
+*[[:File:OKC1.sql.asci|OKC1.sql]]
-*[[Image:OKC2.sql.asci OKC2.sql]]
+*[[:File:OKC2.sql.asci|OKC2.sql]]

@@ Line 6: / Line 6: @@
 There are two SQL script files that were written to process the data:
-*[[Image:OKC1.sql.asci|OKC1.sql]]
+*[[Image:OKC1.sql.asci OKC1.sql]]
-*[[Image:OKC2.sql.asci|OKC2.sql]]
+*[[Image:OKC2.sql.asci OKC2.sql]]

← Older revision		Revision as of 19:55, 19 December 2011
Line 2:		Line 2:

	I will be posting reports and other materials on the OK Cupid project here.		I will be posting reports and other materials on the OK Cupid project here.
		+
		+	==Script Files==
		+
		+	There are two SQL script files that were written to process the data:
		+	*[[Image:OKC1.sql.asci]]
		+	*[[Image:OKC2.sql.asci]]
		+

	==Setup Instructions for Usha==		==Setup Instructions for Usha==

@@ Line 1: / Line 1: @@
-*This page is protected so that only Ed, Rani, Toby and Misiek can read or edit it.
+*This page is protected so that only Ed, Usha, Toby and Misiek can read or edit it.
 I will be posting reports and other materials on the OK Cupid project here.
 ==Data Description from Misiek==

← Older revision		Revision as of 04:19, 30 November 2011
Line 1:		Line 1:
−	*This page is protected so that only Ed, Toby and Misiek can read or edit it.	+	*This page is protected so that only Ed, Rani, Toby and Misiek can read or edit it.

	I will be posting reports and other materials on the OK Cupid project here.		I will be posting reports and other materials on the OK Cupid project here.

@@ Line 56: / Line 56: @@
 ===Gender===
-  +--------+---------+------------+
+  +--------+---------+------------+---------+-----------+---------+-----------+
-  | female | Count   | Percentage |
+  | female | Count   | Percentage | AvgSent | TotalSent | AvgRecd | TotalRecd |
-  +--------+---------+------------+
+  +--------+---------+------------+---------+-----------+---------+-----------+
-  |      0 | 1082104 |     0.5995 |
+  |      0 | 1082104 |     0.5995 |  4.1039 |    339857 |  2.2340 |    185671 |
-  |      1 |  722889 |     0.4005 |
+  |      1 |  722889 |     0.4005 |  3.4872 |    174731 |  2.8786 |    304468 |
-  +--------+---------+------------+
+  +--------+---------+------------+---------+-----------+---------+-----------+
 ===Orientation===
 =Straight, 1=Gay, 2=Bi
-  +-------------+---------+------------+
+  +-------------+---------+------------+---------+-----------+---------+-----------+
-  | orientation | Count   | Percentage |
+  | orientation | Count   | Percentage | AvgSent | TotalSent | AvgRecd | TotalRecd |
-  +-------------+---------+------------+
+  +-------------+---------+------------+---------+-----------+---------+-----------+
-  |           0 | 1570050 |     0.8698 |
+  |           0 | 1570050 |     0.8698 |  3.9727 |    460499 |  2.6087 |    420784 |
-  |           1 |  139697 |     0.0774 |
+  |           1 |  139697 |     0.0774 |  2.9008 |     28802 |  2.1532 |     29212 |
-  |           2 |   95246 |     0.0528 |
+  |           2 |   95246 |     0.0528 |  3.5746 |     25287 |  2.8651 |     40143 |
-  +-------------+---------+------------+
+  +-------------+---------+------------+---------+-----------+---------+-----------+
 ===Ethnicity===
@@ Line 78: / Line 78: @@
 The data was preprocessed to assign individuals who reported more than one gender as being of 'mixed' race. The variable 'ethnicity' is a interger categorization for fast sorting, and the label are provided in the variable 'ethnicitylabels'.
-  +------------------+--------+------------+
+  +------------------+--------+------------+---------+-----------+---------+-----------+
-  | ethnicitylabels  | Count  | Percentage |
+  | ethnicitylabels  | Count  | Percentage | AvgSent | TotalSent | AvgRecd | TotalRecd |
-  +------------------+--------+------------+
+  +------------------+--------+------------+---------+-----------+---------+-----------+
-  | mixed            | 105456 |     0.0584 |
+  | mixed            | 105456 |     0.0584 |  4.0894 |     45674 |  2.8836 |     43540 |
-  | white            | 917178 |     0.5081 |
+  | white            | 917178 |     0.5081 |  3.7588 |    332784 |  2.6029 |    327063 |
-  | black            |  53884 |     0.0299 |
+  | black            |  53884 |     0.0299 |  4.2456 |     18498 |  2.4855 |     13101 |
-  | hispanic_latin   |  61034 |     0.0338 |
+  | hispanic_latin   |  61034 |     0.0338 |  4.2292 |     25278 |  2.9754 |     23348 |
-  | asian            |  54645 |     0.0303 |
+  | asian            |  54645 |     0.0303 |  3.7712 |     16612 |  2.6209 |     17222 |
-  | indian           |  11658 |     0.0065 |
+  | indian           |  11658 |     0.0065 |  4.6518 |      4996 |  2.2567 |      2796 |
-  | middle_eastern   |   5916 |     0.0033 |
+  | middle_eastern   |   5916 |     0.0033 |  6.0000 |      3282 |  2.9282 |      1918 |
-  | native_american  |   4056 |     0.0022 |
+  | native_american  |   4056 |     0.0022 |  5.0511 |      1682 |  3.0157 |      1345 |
-  | pacific_islander |   3238 |     0.0018 |
+  | pacific_islander |   3238 |     0.0018 |  3.6478 |      1098 |  2.9398 |      1173 |
-  | other            |  24761 |     0.0137 |
+  | other            |  24761 |     0.0137 |  4.3936 |     11454 |  2.9021 |      9664 |
-  | none             | 563167 |     0.3120 |
+  | none             | 563167 |     0.3120 |  3.9094 |     53230 |  2.1889 |     48969 |
-  +------------------+--------+------------+
+  +------------------+--------+------------+---------+-----------+---------+-----------+
 The distribution for the reported number of ethnicities (with >1 assigned to mixed) is:
@@ Line 118: / Line 118: @@
 *[http://en.wikipedia.org/wiki/List_of_ZIP_code_prefixes List of Zip Code Prefixes]
-  +------+--------+------------+
+  +------+--------+------------+---------+-----------+---------+-----------+
-  | zip1 | Count  | Percentage |
+  | zip1 | Count  | Percentage | AvgSent | TotalSent | AvgRecd | TotalRecd |
-  +------+--------+------------+
+  +------+--------+------------+---------+-----------+---------+-----------+
-  | 0    | 178253 |     0.0988 |
+  | 0    | 178253 |     0.0988 |  3.8582 |     55219 |  2.5989 |     52293 |
-  | 1    | 229719 |     0.1273 |
+  | 1    | 229719 |     0.1273 |  4.4691 |     84690 |  2.6089 |     73503 |
-  | 2    | 136538 |     0.0756 |
+  | 2    | 136538 |     0.0756 |  3.7811 |     38575 |  2.5554 |     37794 |
-  | 3    | 148748 |     0.0824 |
+  | 3    | 148748 |     0.0824 |  3.8744 |     41979 |  2.6477 |     40292 |
-  | 4    | 126115 |     0.0699 |
+  | 4    | 126115 |     0.0699 |  3.6275 |     33953 |  2.5882 |     33064 |
-  | 5    |  66067 |     0.0366 |
+  | 5    |  66067 |     0.0366 |  3.3721 |     15643 |  2.3133 |     15157 |
-  | 6    | 105097 |     0.0582 |
+  | 6    | 105097 |     0.0582 |  3.6479 |     29650 |  2.5607 |     29468 |
-  | 7    | 126851 |     0.0703 |
+  | 7    | 126851 |     0.0703 |  4.0350 |     38615 |  2.8075 |     37758 |
-  | 8    |  84204 |     0.0467 |
+  | 8    |  84204 |     0.0467 |  3.7276 |     23875 |  2.6178 |     23128 |
-  | 9    | 322186 |     0.1785 |
+  | 9    | 322186 |     0.1785 |  3.8279 |    103851 |  2.7357 |    103539 |
-  | 99   | 281215 |     0.1558 |
+  | 99   | 281215 |     0.1558 |  3.6252 |     48538 |  2.2509 |     44143 |
-  +------+--------+------------+
+  +------+--------+------------+---------+-----------+---------+-----------+
 ===Age===
@@ Line 138: / Line 138: @@
 The age ranges were created arbitrarily - though they have worked out reasonable well. Further refinement is possible. The variable 'age_rnum' is a integer categorization, and 'age_range' provides the variable. Ages were calculated using 'birth_year' using 2010 as the reference point.
-  +-----------+--------+------------+
+  +-----------+--------+------------+---------+-----------+---------+-----------+
-  | age_range | Count  | Percentage |
+  | age_range | Count  | Percentage | AvgSent | TotalSent | AvgRecd | TotalRecd |
-  +-----------+--------+------------+
+  +-----------+--------+------------+---------+-----------+---------+-----------+
-  | 15-19     |  97659 |     0.0541 |
+  | 15-19     |  97659 |     0.0541 |  4.8070 |     30385 |  3.5139 |     35708 |
-  | 20-24     | 485028 |     0.2687 |
+  | 20-24     | 485028 |     0.2687 |  3.9309 |    146882 |  2.8494 |    159322 |
-  | 25-29     | 475376 |     0.2634 |
+  | 25-29     | 475376 |     0.2634 |  3.5992 |    131205 |  2.4778 |    128834 |
-  | 30-34     | 274281 |     0.1520 |
+  | 30-34     | 274281 |     0.1520 |  4.4587 |     89759 |  2.4158 |     66236 |
-  | 35-39     | 155925 |     0.0864 |
+  | 35-39     | 155925 |     0.0864 |  3.7430 |     42681 |  2.3843 |     36490 |
-  | 40-44     | 109121 |     0.0605 |
+  | 40-44     | 109121 |     0.0605 |  3.5817 |     27776 |  2.4255 |     24939 |
-  | 45-49     |  78618 |     0.0436 |
+  | 45-49     |  78618 |     0.0436 |  3.6452 |     19225 |  2.2995 |     16338 |
-  | 50-59     |  95810 |     0.0531 |
+  | 50-59     |  95810 |     0.0531 |  3.3421 |     21055 |  2.1347 |     17831 |
-  | >60       |  33175 |     0.0184 |
+  | >60       |  33175 |     0.0184 |  2.9332 |      5620 |  1.8930 |      4441 |
-  +-----------+--------+------------+
+  +-----------+--------+------------+---------+-----------+---------+-----------+
 Birth year is self-reported almost surely wrongly so in some case. It ranges from 1900 to 1995.
@@ Line 158: / Line 158: @@
 The account age range variables, 'acc_age_rnum' for the interger categorization and 'acc_age_range' for the labels, were created arbitrarily. These could easily be refined, but were created to examine differences in messaging and viewing activitiy - and to provide a normalization base. The oldest account is 2,613 days. The youngest is 0 days.
-  +---------------+--------+------------+
+  +---------------+--------+------------+---------+-----------+---------+-----------+
-  | acc_age_range | Count  | Percentage |
+  | acc_age_range | Count  | Percentage |         |           |         |           |
-  +---------------+--------+------------+
+  +---------------+--------+------------+---------+-----------+---------+-----------+
-  | 0             |   1385 |     0.0008 |
+  | 0             |   1385 |     0.0008 |         |           |         |           |
-  | 1             |   7914 |     0.0044 |
+  | 1             |   7914 |     0.0044 |         |           |         |           |
-  | 2             |   7967 |     0.0044 |
+  | 2             |   7967 |     0.0044 |         |           |         |           |
-  | 3             |   9051 |     0.0050 |
+  | 3             |   9051 |     0.0050 |         |           |         |           |
-  | 4             |   9134 |     0.0051 |
+  | 4             |   9134 |     0.0051 |         |           |         |           |
-  | 5             |  10032 |     0.0056 |
+  | 5             |  10032 |     0.0056 |         |           |         |           |
-  | 6-10          |  35862 |     0.0199 |
+  | 6-10          |  35862 |     0.0199 |         |           |         |           |
-  | 11-15         |  39986 |     0.0222 |
+  | 11-15         |  39986 |     0.0222 |         |           |         |           |
-  | 16-30         | 136983 |     0.0759 |
+  | 16-30         | 136983 |     0.0759 |         |           |         |           |
-  | 31-60         | 233560 |     0.1294 |
+  | 31-60         | 233560 |     0.1294 |         |           |         |           |
-  | 61-100        | 237369 |     0.1315 |
+  | 61-100        | 237369 |     0.1315 |  5.2286 |    142694 |  3.5119 |    144483 |
-  | 101-365       | 535715 |     0.2968 |
+  | 101-365       | 535715 |     0.2968 |  3.5052 |    220052 |  2.4617 |    220381 |
-  | 366-3650      | 540035 |     0.2992 |
+  | 366-3650      | 540035 |     0.2992 |  3.5435 |    151842 |  2.1519 |    125275 |
-  +---------------+--------+------------+
+  +---------------+--------+------------+---------+-----------+---------+-----------+
 ===Other variables===