Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

12179 create Answers table and migrate existing response data to populate it #966

Open
wants to merge 32 commits into
base: develop
Choose a base branch
from

Conversation

DevneyHamilton
Copy link
Contributor

@DevneyHamilton DevneyHamilton commented May 2, 2022

Business planning loads slowly because for over 300 questions and their corresponding answers, Madeline has to extract question and answer data from the database, associate answers with questions, assign required-or-not status to questions based on the loan type, render the question/answer heirarchy into html, and then make javascript adjustments to every question and answer in the browser. The way to reduce loading time is to load fewer questions and answers at once. Loading these questions in smaller portions requires:
Phase 1: replace a single json blob holding all 300+ answers in one database cell with a database table where each answer has its own row (engineering done, needs careful release)
Phase 2: replace an unconventional saving mechanism that can’t be adjusted with conventional Rails form saving (engineering done, needs careful release)
Phase 3: divide the existing credit memo questionnaire such that each top-level section can be saved and loaded separately (engineering in progress). Maintain all legacy data through this change.

This PR represents a 'first phase' of of the three above. In this phase:

  • there's a new Answer model and table. An Answer belongs to a response set and question. It stores the answer to that question in that response set. (Response sets, as before, belong to a loan and a question_set).
  • there's a 'one time changes' rake task that copies response sets' existing 'custom_data' json blob into a new Answers table
  • answer information is read from the Answers table instead of custom_data in response_sets.
  • answers coming in from the questionnaire form are saved BOTH to the new Answers table and the custom_data on response sets.

The next phase will involve:

  • removing response_sets' custom_data field all together
  • removing question's internal_name field
  • removing the method_missing override in response_set.rb that handles saving answers from the questionnaire form
  • making the questionnaire form work with nested_attributes for answers (probably; if there are issues with this, i may move to json api style in anticipation of phase 3)
  • removing Response model; any still needed functionality distributed among response set, question set, question, and answer

Phase 3:

  • pending design work, change how questionnaires display and are saved to, mainly so that users can interact with a smaller number of questions & answers at a time, reducing load times and possibly simplifying the front end experience.

Future:
This move to answers table instead of json blob, and accompanying clean up, opens up the possibility for modernizing the form system to have a responsive frontend (e.g. with React) and an API.

…an, and editing answers works, using answers table, needs auto testing
…ing between answer and old style json for not applicable and boolean data type questions
… support flag about whether to include text like numeric answers
… a problem that needs to be fixed next. pause here to try changing form to use question id.
…reating "field_" internal names before id is available
…e info only calculating in question editing, and not in questionnaire loading
@@ -74,9 +50,10 @@ def questions
end

def question_sets
# We want self to come first for deterministic behavior in specs. After that it doesn't really matter.
# self_and_descendants orders by depth so we are good.
division.self_and_descendants.flat_map { |d| QuestionSet.where(division: d).order(:kind).to_a }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the previous version of this meant that if one is in a division that inherits question set(s) from ancestor divisions when making a data export, those inherited question sets were skipped.

# Conflicts:
#	db/schema.rb
#	lib/tasks/one_time_changes.rake
…roup questions must contain string 'field' to identify form fields that need to be saved as answers.
…r migrations already deployed to staging and prod
@DevneyHamilton DevneyHamilton changed the title 12179 answers migration 12179 create Answers table and migrate existing data Jun 15, 2023
@DevneyHamilton DevneyHamilton changed the title 12179 create Answers table and migrate existing data 12179 create Answers table and migrate existing response data to populate it Jun 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant