Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix bad Washington files #212

Closed
ghing opened this issue Oct 8, 2014 · 6 comments
Closed

Fix bad Washington files #212

ghing opened this issue Oct 8, 2014 · 6 comments
Assignees
@ghing
Labels
in progress WA
Milestone
2014-10-13

Comments

@ghing
Copy link
Contributor

ghing commented Oct 8, 2014

@EricLagerg clearly documented files that can't be loaded because they seem to contain HTML error page output or that are unwieldy Excel spreadsheet. Try to fix the URLs (likely in the datasource) so that real data can be downloaded and explicitly document the nasty Excel files.
@ghing ghing added the WA label Oct 8, 2014
@ghing ghing self-assigned this Oct 8, 2014
@ghing ghing added this to the 2014-10-06 milestone Oct 8, 2014
@ghing
Copy link
Contributor Author

ghing commented Oct 9, 2014

These CSV files all have the same problem:

20090818__wa__primary__pierce__county.csv
20090818__wa__primary__ferry__county.csv
20090818__wa__primary__wahkiakum__county.csv
20090818__wa__primary__whatcom__county.csv
20090818__wa__primary__pend_oreille__county.csv
20090818__wa__primary__kitsap__county.csv
20090818__wa__primary__kittitas__county.csv

Output stored in the file says:

<html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="%2fresults%2f20090818%2fexport%2f20090818_Kittitas.csv">here</a>.</h2>
</body></html>

When I try to grab the URL with wget, it seems like there's a redirect loop:

wget http://vote.wa.gov/results/20090818/export/20090818_Pierce.csv
--2014-10-08 18:56:17--  http://vote.wa.gov/results/20090818/export/20090818_Pierce.csv
Resolving vote.wa.gov (vote.wa.gov)... 64.146.248.150
Connecting to vote.wa.gov (vote.wa.gov)|64.146.248.150|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location: /results/20090818/export/20090818_Pierce.csv [following]
--2014-10-08 18:56:18--  http://vote.wa.gov/results/20090818/export/20090818_Pierce.csv
Reusing existing connection to vote.wa.gov:80.
HTTP request sent, awaiting response... 302 Found
Location: /results/20090818/export/20090818_Pierce.csv [following]
--2014-10-08 18:56:18--  http://vote.wa.gov/results/20090818/export/20090818_Pierce.csv
Reusing existing connection to vote.wa.gov:80.
HTTP request sent, awaiting response... 302 Found
...
20 redirections exceeded.

I'm going to check and make sure there aren't different URLs that are accessible from website, but otherwise, it seems like the path forward on these is to contact the Washington Secretary of State and let them know their webserver is misconfigured.

@ghing
Copy link
Contributor Author

ghing commented Oct 9, 2014

I took a look at http://vote.wa.gov/results/20090818/Export.html and found that there aren't entries for any of the counties corresponding to the files listed above:

  • Pierce
  • Ferry
  • Wahkiakum
  • Whatcom
  • Pend Oreille
  • Kitsap
  • Kittitas

@ghing
Copy link
Contributor Author

ghing commented Oct 9, 2014

Emailed Nick about the missing counties.

@ericlagergren
Copy link
Member

Perhaps I should talk to my boss about sponsoring a bill that reforms election data in Washington state?

Anyway, should we try to find a way to transform the unwieldy Excel data before we load it into the db? It'd definitely have to be on a per-file basis, as some of the county auditors had a little way too much creative license with the file formatting.

@ghing
Copy link
Contributor Author

ghing commented Oct 10, 2014

I got a response from Nick about those missing CSVs from 2009:

Those counties did not hold a Primary in 2009. (Kitsap County did not participate in our elections
database at that time, so there may have been local contests there that were not entered into the
database. The same is true for King and Yakima Counties, but they do appear in the results
because they had state-level contests that year.)

The only state-level contests on the Primary ballot in 2009 were 3 unexpired legislative terms and
one unexpired Court of Appeals term. Local contests do not have a Primary unless at least two
candidates file.

I'll update the datasource to not generate mappings for these counties for this election.

ghing added a commit that referenced this issue Oct 11, 2014
@ghing
Don't create mappings for counties that don't have results
9d65696 
Don't create mappings for counties that didn't produce results
in the 2009 Washington Primary election.

Addresses #212
@ghing ghing modified the milestones: 2014-10-06, 2014-10-13 Oct 14, 2014
@ghing
Copy link
Contributor Author

ghing commented Oct 17, 2014

Closing. The datasource has been updated to not create mappings for the counties that don't have results for a particular election. I've created a separate issues for the results that need one-off loader classes as that's going to be a long, slow row to hoe. The issue is #225.

@ghing ghing closed this as completed Oct 17, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ghing ghing

Labels
in progress WA
Projects
None yet
Milestone
2014-10-13
Development

No branches or pull requests
2 participants
@ghing @ericlagergren