Code debt #68
Merged
Code debt #68
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
the computation is pretty repetitive (it happens on every run), so cache the tables back to the current hdf store so you don't have to do all the data tweaking every time
still a work in progress
using buildings table for both estimation and simulation
many major preprocessing steps were performed when tables were accessed. this is a major benefit of orca - you can write transformations to data tables in code and perform them every time the table is accessed, and share the code via github. there are limits to this, which mainly have to do with processing time. for longer running processes, it's not efficient to run every time a table is accessed in order to do interactive exploratory work. there's no hard and fast rule for what crosses, the line, but for deed restricted units and adjustments of vacancy rates, we had probably crossed it. there is now a "preprocessing" MODE in run.py which runs the steps which currently fall under this rubric, as well as a preprocessing.py file where the code lives. when you get the .h5 file from s3, you will need to run this mode before running anything else. as part of this change, run.py now has a proper usage statement using argparse to get arguments. the "--mode preprocessing" argument is used to run preprocessing.
also modified parcel_is_allowed_func and a few other things
…-bldg-id Drop bldg ids
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
numerous aspects of code debt payment including