FailedChanges

Summary

  1. Initial version of pipeline reset enhancements (commit: f062c984135fae5db8cd3a075fc56fbbf1984cf6) (details)
  2. Return the correct field for querying smoothing entries (commit: 36815d8533edfc0d86406f13a85098b6bfe67684) (details)
  3. Refactor the bulk of the reset logic into a separate module (commit: 489ecf3bb17c725e2df18317efed980cfc411ea2) (details)
  4. Restore the code to clear the newly added entries at the end of the (commit: 26efe597c2b71332bf45cd20833533b4a9fe0858) (details)
  5. Test and fix the reset code (commit: 7cef871f390dfb8648c7840dbdbfab7068812dc1) (details)
Commit f062c984135fae5db8cd3a075fc56fbbf1984cf6 by shankari
Initial version of pipeline reset enhancements
Comments describe current thoughts around fixes.
https://github.com/e-mission/e-mission-server/issues/333
Remaining fixes:
- Since we find the last entry for each timeline and reset it, the
reset
methods have to run on one user at a time, even for "all"
- Our query returns the first place that is deleted. The place before
that is
the one that we need to reset. Work through differences between
overlapping
and non-overlapping trips on finding this and resetting correctly.
(commit: f062c984135fae5db8cd3a075fc56fbbf1984cf6)
The file was modifiedbin/reset_pipeline.py (diff)
Commit 36815d8533edfc0d86406f13a85098b6bfe67684 by shankari
Return the correct field for querying smoothing entries
The entries are smoothed by querying for sections and then smoothing
the sections. As part of 96ed8711135f7d4cae0706d5079c8dbff8a02b86, we
converted the queries to work on any field instead of fields within
data, which meant that the fields had to be specified as `"data.ts"`
instead of `"ts"`. We changed the field for `SECTION_SEGMENTATION` to
`data.end_ts`, but didn't change the query for the smoothing. This
meant that we have not been running smoothing properly since Apr.
This resolves https://github.com/e-mission/e-mission-server/issues/334
(commit: 36815d8533edfc0d86406f13a85098b6bfe67684)
The file was modifiedemission/storage/pipeline_queries.py (diff)
Commit 489ecf3bb17c725e2df18317efed980cfc411ea2 by shankari
Refactor the bulk of the reset logic into a separate module
To make it easier for unit testing, and to give a better structure to
the code
(commit: 489ecf3bb17c725e2df18317efed980cfc411ea2)
The file was addedemission/pipeline/reset.py
The file was modifiedbin/reset_pipeline.py (diff)
Commit 26efe597c2b71332bf45cd20833533b4a9fe0858 by shankari
Restore the code to clear the newly added entries at the end of the
test
While this is not strictly required since we create a new UUID for each
run, it is just Good Practice to clean up the database after we create
a bunch of entries. We commented this out to make it easier to debug,
but then we should make that change locally instead...
(commit: 26efe597c2b71332bf45cd20833533b4a9fe0858)
The file was modifiedemission/tests/analysisTests/intakeTests/TestPipelineRealData.py (diff)
Commit 7cef871f390dfb8648c7840dbdbfab7068812dc1 by shankari
Test and fix the reset code
Add a new test case for the resetting, and update the reset code to
make the tests to pass.
Details of the various stages of getting it to pass are at
https://github.com/e-mission/e-mission-server/issues/333
(commit: 7cef871f390dfb8648c7840dbdbfab7068812dc1)
The file was addedemission/tests/pipelineTests/TestPipelineReset.py
The file was modifiedemission/pipeline/reset.py (diff)
The file was modifiedemission/storage/decorations/place_queries.py (diff)