Code

A3C LSTM GA for language grounding#7

Code

Open

beduffy wants to merge 98 commits intomasterTheMTank/cups-rl:masterfrom A3C_LSTM_GA_barebonesTheMTank/cups-rl:A3C_LSTM_GA_barebonesCopy head branch name to clipboard

+5,522-432

Commits

Commits on Sep 29, 2018

• Added bounding_boxes_and_test_reward.py which allows experimentation with bounding boxes and visualising them and checking if we are close and focusing on an object

  Show description for 4554cd6
  beduffy

  committed
  4554cd6

  Copy full SHA for 4554cd6

Commits on Sep 30, 2018

• Added microwave or mug navigation task to wrapper

  Show description for afd03a3
  beduffy

  committed
  afd03a3

  Copy full SHA for afd03a3

• Fixed bounding boxes to use instance_detections2D attribute instead of colorBounds

  Show description for 0c84638
  beduffy

  committed
  0c84638

  Copy full SHA for 0c84638

• Fixed "class and object info not showing in metadata on reset of env"

  Show description for 12a8e9f
  beduffy

  committed
  12a8e9f

  Copy full SHA for 12a8e9f

• Started monitoring episode lengths (3rd figure) and time taken for microwave task

  Show description for 8e16277
  beduffy

  committed
  8e16277

  Copy full SHA for 8e16277

• Added dense reward option for microwave task

  Show description for 81c7069
  beduffy

  committed
  81c7069

  Copy full SHA for 81c7069

Commits on Oct 6, 2018

• Added Devendra Chaplot's DeepRL grounding A3C model

  beduffy

  committed
  66d5915

  Copy full SHA for 66d5915

• Added natural language option to environment with 1 task

  Show description for 072e754
  beduffy

  committed
  072e754

  Copy full SHA for 072e754

• Added many changes! added todos, added training of A3C_GA version

  Show description for f84b440
  beduffy

  committed
  f84b440

  Copy full SHA for f84b440

Commits on Oct 7, 2018

• Put all plots into utils common function and used for both models.

  Show description for 1d9b78c
  beduffy

  committed
  1d9b78c

  Copy full SHA for 1d9b78c

• Added checkpointing and resuming in experiment folder

  Show description for b86e5a1
  beduffy

  committed
  b86e5a1

  Copy full SHA for b86e5a1

• Added script to read latest 3 plots saved in png

  Show description for a2ce6e8
  beduffy

  committed
  a2ce6e8

  Copy full SHA for a2ce6e8

• Added entity features into state information. 27 features. Created new model to deal with it.

  Show description for 94d193c
  beduffy

  committed
  94d193c

  Copy full SHA for 94d193c

Commits on Oct 11, 2018

• Added two sentence tasks (turn left or turn right 3 times)

  Show description for 75f64d7
  beduffy

  committed
  75f64d7

  Copy full SHA for 75f64d7

Commits on Oct 16, 2018

• Removed reward clamp Saved action probabilities plot every episode

  beduffy

  committed
  1a5bd49

  Copy full SHA for 1a5bd49

• Added go and look and microwave and cup tasks and test case to check

  Show description for 90bd31b
  beduffy

  committed
  90bd31b

  Copy full SHA for 90bd31b

Commits on Oct 19, 2018

• Fixed massive bug of not loading new sentence and much more

  Show description for 9a02df5
  beduffy

  committed
  9a02df5

  Copy full SHA for 9a02df5

• Added single word instruction sentences and fixed a bug with current object type

  Show description for 828d950
  beduffy

  committed
  828d950

  Copy full SHA for 828d950

Commits on Oct 21, 2018

• Added check for empty arrays for plotting newer graphs

  Show description for fb1ddcc
  beduffy

  committed
  fb1ddcc

  Copy full SHA for fb1ddcc

• Tried LR 0.0003, Added experiments text file, saved episode number

  Show description for b212284
  beduffy

  committed
  b212284

  Copy full SHA for b212284

Commits on Oct 24, 2018

• Added checkpoint and first experiment folder. Fixed latest plot blug.

  Show description for a5cebf2
  beduffy

  committed
  a5cebf2

  Copy full SHA for a5cebf2

Commits on Oct 28, 2018

• Added multi-threading, save total length/number of episodes, counter, timing and benchmarking

  Show description for 9977a2a
  beduffy

  committed
  9977a2a

  Copy full SHA for 9977a2a

Commits on Nov 1, 2018

• Fixed checkpoint_counter for new experiment, some todos and comments

  beduffy

  committed
  7b8b87f

  Copy full SHA for 7b8b87f

• Added .gitignore for experiments

  beduffy

  committed
  cb25429

  Copy full SHA for cb25429

Commits on Nov 4, 2018

• Add Rainbow DQN code

  Show description for a43ea64
  fernandotorch

  committed
  a43ea64

  Copy full SHA for a43ea64

Commits on Nov 6, 2018

• Some comments, style and prognostications

  beduffy

  committed
  324a0f0

  Copy full SHA for 324a0f0

Commits on Nov 17, 2018

• Merge branch 'A3C_barebones' into Rainbow-DQN

  fernandotorch

  committed
  1e64989

  Copy full SHA for 1e64989

• Fix channel dimension from state

  Show description for 44da993
  fernandotorch

  committed
  44da993

  Copy full SHA for 44da993

• Merge branch 'A3C_LSTM_GA_two_sentence' into A3C_LSTM_GA_barebones

  Show description for b419713
  beduffy

  committed
  b419713

  Copy full SHA for b419713

• Added A3C GA model to model.py and removed 3 files I won't need (good-experiment-ids.txt, model.py, check_latest_plots)

  Show description for 3acecaf
  beduffy

  committed
  3acecaf

  Copy full SHA for 3acecaf

Commits on Jan 29, 2019

• Import A3C_LSTM, added natural language param, some copying into train.py file but commented natural language (unfinished), beginnings of natural language task

  Show description for 3e52ba7
  beduffy

  committed
  3e52ba7

  Copy full SHA for 3e52ba7

• Added cameraY, gridSize, incremental_rotation_mode with 10 degree rotations params. Loading build from unity. Rotation broke

  beduffy

  committed
  1dd0a72

  Copy full SHA for 1dd0a72

Commits on Jan 30, 2019

• Added new task example for agent on the ground with cups. Added cameraY, gridSize, rotation, build_path to config_example.json. set reasonable defaults and checks

  beduffy

  committed
  89bc411

  Copy full SHA for 89bc411

Commits on Jan 31, 2019

• Added continuous mode to initialise of ai2thor which finally allowed moving in any direction

  beduffy

  committed
  b751b27

  Copy full SHA for b751b27

Commits on Feb 2, 2019

• Added argparse to continuous example for build path, default false incremental rotation, fixed 1 test by gridSize 0.25, both to readme config.json example

  beduffy

  committed
  1bb10e9

  Copy full SHA for 1bb10e9

• NLP instructions working. Added natural_language if/else statements, cleaned up NaturalLanguageLookAtTask and NL environment settings, removed batch forgotten if statements, a3c_ga to float, many n…

  Show description for 11fa2c4
  beduffy

  committed
  11fa2c4

  Copy full SHA for 11fa2c4

Commits on Feb 3, 2019

• Merge branch 'master' of https://github.com/TheMTank/ai2thor-experiments into A3C_LSTM_GA_barebones

  beduffy

  committed
  23bff3a

  Copy full SHA for 23bff3a

• Renamed to Bowl and Mug, passed curr_object_type in event to task, expand_as, euclidean distance for close_enough

  Show description for e7e2d30
  beduffy

  committed
  e7e2d30

  Copy full SHA for e7e2d30

• Added uuid eid args and old checkpointing, todos, references to old commented ways to delete in next commit, timing, todos, printing of policy/value loss

  beduffy

  committed
  9feb73f

  Copy full SHA for 9feb73f

• save_checkpoint function with os paths (called in args num_steps), cleared commented checkpoint code and tested, moved bounding_boxes code to tests

  beduffy

  committed
  318e54f

  Copy full SHA for 318e54f

• Deleted 8 old files. Old wrapper finally gone (entity feats within if needed). Old A3C thread stats gone in main.py. Old train.py with some nice comments on how to split up and standardise common R…

  Show description for 6889f30
  beduffy

  committed
  6889f30

  Copy full SHA for 6889f30

• Added option to disable lookupdown actions, added option to do num random actions on reset, fixed utils read config bug and saved latest argparse params to json file in experiment folder

  beduffy

  committed
  c133832

  Copy full SHA for c133832

Commits on Feb 5, 2019

• Added NaturalLanguageLookAtObjectTask, NaturalLanguageNavigateToObjectTask, nlpbase task, removed all natural language from Env with modified step function with extra state info from task, task_uti…

  Show description for 14379ad
  beduffy

  committed
  14379ad

  Copy full SHA for 14379ad

Commits on Feb 6, 2019

• Added a new large test case for naturalLanguageLookAtTask which tests that 5 different objects work and added env.close() to end of many test cases

  beduffy

  committed
  011332e

  Copy full SHA for 011332e

• Added two examples, inbuilt_interactive_mode.py with argparse for unity build and 2nd was renamed bounding_boxes file. Removed old functions in file and use gym_task_utils. Added line option from b…

  Show description for 2d13a8f
  beduffy

  committed
  2d13a8f

  Copy full SHA for 2d13a8f

• Removed natural_language_instructions from config since it's decided by task, check if reset before first step and raise ValueError otherwise, random seed

  beduffy

  committed
  97fbe3a

  Copy full SHA for 97fbe3a

• Moved show_bounding_boxes() to task_utils.py with extra features, raise ValueError if objectType not found, pass 2d+3d distance thresholds to lookAt reward functions. Added verbose option for debug…

  Show description for 6d2eb98
  beduffy

  committed
  6d2eb98

  Copy full SHA for 6d2eb98

• Fixed merge conflicts from merging cozmo_env. ai2thor_env.py

  beduffy

  committed
  90932b0

  Copy full SHA for 90932b0

• Made sure grayscale on/off worked, added args.task_name, removed all mentions of args.natural_language, added todos, turn_instruction_str_to_tensor to task_utils.py, apple and mug default, controll…

  Show description for 97743c7
  beduffy

  committed
  97743c7

  Copy full SHA for 97743c7

Commits on Feb 8, 2019

• Added writer to train and saved scalars into tensorboard, added cozmo env params (gridsize, camerayY, buildpath), added and removed todos, max_episode_length to config_example.json, pass whole conf…

  Show description for 655d9b1
  beduffy

  committed
  655d9b1

  Copy full SHA for 655d9b1

Commits on Feb 10, 2019

• Fixed max_episode_length bug and passed kwargs correctly all the way to BaseTask, renamed string to PickUpTask, renamed file without on_ground, gridSize 0.05,

  beduffy

  committed
  ee3a9eb

  Copy full SHA for ee3a9eb

• Added NaturalLanguagePickUpObjectTask, Added movement_reward, incremental_rotation, saved all_scalars to experiment folder, saved episode_number and episde length correctly in tensorboard

  beduffy

  committed
  83ec524

  Copy full SHA for 83ec524

• Added 3 extra config files (for specific task variations) and renamed to default_config.json

  beduffy

  committed
  d033b34

  Copy full SHA for d033b34

• Fixed test.py for both A3C+A3C_GA, added negative reward for picking up wrong object and added comments to pdb_interactive

  beduffy

  committed
  2138ac9

  Copy full SHA for 2138ac9

• Fixed reset bug by setting step_num to 0, changed to build_file_name in build_files folder, added moveupdown_actions option

  beduffy

  committed
  a51106b

  Copy full SHA for a51106b

Commits on Feb 11, 2019

• Added calculate_lstm_input_size_for_A3C_LSTM_GA with hardcoded factors and used it, docstring to A3C_LSTM_GA and inputted vocab_size, episode_length. Added todos, comments

  beduffy

  committed
  a6eeeaf

  Copy full SHA for a6eeeaf

• Changed task random walk example to take build_file_name and auto-calculated x_attention dimensions by changing calculate_input_width_height func to return tuple

  beduffy

  committed
  5507373

  Copy full SHA for 5507373

Commits on Feb 12, 2019

• Fixed off by one episode_length time embedding by incrementing after step, moved 3 ai2thor tests to ai2thor_examples.py, default_config PickUpTask and movement_reward in others,

  beduffy

  committed
  cea21a3

  Copy full SHA for cea21a3

Commits on Feb 13, 2019

• Added test case for NaturalLanguagePickUpObjectTask, removed 3 ai2thor test cases and made them examples, added config file name and checkpoint-freq arg params, saved latest_config.json and removed…

  Show description for 2ffe6f3
  beduffy

  committed
  2ffe6f3

  Copy full SHA for 2ffe6f3

Commits on Feb 16, 2019

• Renamed incremental_rotation to continuous_movement, added a few comments, startswith+endswith for Object interactions

  beduffy

  committed
  d12e6c5

  Copy full SHA for d12e6c5

Commits on Feb 17, 2019

• Added todos, printed elapsed training time and time for 20 steps only every 100 backprops, distance_threshold_3d to task config for nl_lookat task

  beduffy

  committed
  0e167f2

  Copy full SHA for 0e167f2

Commits on Feb 18, 2019

• Added num-random-actions-at-init to argparse, max_episode_length back, added comments, commented word_embedding expand (must understand), added todos, saved image correctly, print predicted value f…

  Show description for eca78bf
  beduffy

  committed
  eca78bf

  Copy full SHA for eca78bf

Commits on Feb 19, 2019

• Added verbose-num-steps arg, printed tensorboard command, shutil import and model_is_best param, added avg_rewards with avg_over_num_episodes, fixed image tensorboard bug and reordered some code

  beduffy

  committed
  9e4ba68

  Copy full SHA for 9e4ba68

Commits on Feb 20, 2019

• Cleaned comments, removed some todos, changed to floorplan2 in rotate only, absolute rotation to 0.0 on reset,

  beduffy

  committed
  132df1b

  Copy full SHA for 132df1b

Commits on Feb 21, 2019

• Cleaned+added comments and removed some todos

  beduffy

  committed
  de96042

  Copy full SHA for de96042

Commits on Feb 22, 2019

• Fixed multi-word sentence bug, rotate_only will have multi-word sentences since otherwise too easy, removed final todos in model.py

  beduffy

  committed
  6edc1e3

  Copy full SHA for 6edc1e3

Commits on Feb 23, 2019

• Removed todos regarding predicted value spike (didn't find reason), removed rainbow files so PR looks neater

  beduffy

  committed
  97eada7

  Copy full SHA for 97eada7

• Fix merge conflicts from merging cozmo_env into A3C_GA. continuous_movement instead of incremental

  beduffy

  committed
  64e57e1

  Copy full SHA for 64e57e1

• Fixed test() function, renamed file, added many comments, fixed NL_pickup task and turned off lookupdown_actions, if open_close check for allowed_objects, allow to specify pickup_crosshair_distance…

  Show description for b29c93e
  beduffy

  committed
  b29c93e

  Copy full SHA for b29c93e

• Renamed avg_rewards to avg_episode_returns, added euclidean distance check to pickup because it was too large as default, created calculate_euc_distance_between_agent_and_object() function,

  beduffy

  committed
  7668186

  Copy full SHA for 7668186

Commits on Feb 24, 2019

• Added 4 vizdoom_data files, 1 vizdoom_maps room.wad file, 3 python vizdoom_utils files and added GroundingEnv into envs.py with comment splitting between Atari

  beduffy

  committed
  ce4dd9a

  Copy full SHA for ce4dd9a

• Renamed envs to env_atari.py and took out vizdoom part into env_vizdoom.py. Movement reward set to -0.001 and pickup euclidean distance to 0.3

  beduffy

  committed
  8a8b1a3

  Copy full SHA for 8a8b1a3

• Got VizDoom working in main.py and model.p.y Added ViZDoom argparams, better experiment-id with date+uuid if not specified, inputted resolution to model and calculated non-square lstm_cell_size and…

  Show description for ea83ca5
  beduffy

  committed
  ea83ca5

  Copy full SHA for ea83ca5

• Added random scenes on reset option within task, build_bowls_vs_cups_fp1_v_0.1 config added (first versioned build), ai2thor_inbuilt_interactive_mode broken and todos

  beduffy

  committed
  09e052d

  Copy full SHA for 09e052d

• Added a3c/main.py variants to README.md with more info on natural language tasks. Added seed() with warning saying not implemented to GroundingEnv so it does not crash, renamed max_object_pickup vars

  beduffy

  committed
  7bde258

  Copy full SHA for 7bde258

Commits on Feb 26, 2019

• Got train and test working in vizdoom, added beautiful unpack_state() function to task_utils.py and used 3 times in train, renamed to image_state, passed resolution to models and added 2d euclidean…

  Show description for 7796fd7
  beduffy

  committed
  7796fd7

  Copy full SHA for 7796fd7

• Got VizDoom running from any folder by converting relative paths to absolute paths for data, changed vizdoom reset() to only return 1 thing from 4, seed function fix

  beduffy

  committed
  936027f

  Copy full SHA for 936027f

• Fixed glob for latest checkpoint, fixed ai2thor inbuilt interactive by clearly saying terminal only, check if unity build file path exists in two places

  beduffy

  committed
  1a1548a

  Copy full SHA for 1a1548a

Commits on Feb 27, 2019

• Added complicated logic for args.resume-latest-config (default is no override allowed from config file), reordered main.py and more comments, Fixed missing R problem, added 4 room config file, fixe…

  Show description for 2b83816
  beduffy

  committed
  2b83816

  Copy full SHA for 2b83816

Commits on Feb 28, 2019

• Removed args.task-name, removed Exception as e, tensorboard creation to bottom, removed microwave or cup checkpoint, added todo to utils and a bit more info to README.md about config+task variants

  beduffy

  committed
  cdf2df9

  Copy full SHA for cdf2df9

• Added NaturalLanguagePickUpMultipleObjectTask with no terminal with config file, Added ability to remove put_interaction alone, allowed multiple objects in inventory in wrapper and in build file C#…

  Show description for d1a89c0
  beduffy

  committed
  d1a89c0

  Copy full SHA for d1a89c0

Commits on Mar 2, 2019

• Added num_backprops to checkpoint and reload, checked for "and not" pickup_interaction as well, only add scalar for value+policy loss for rank == 0 in train

  beduffy

  committed
  5569f3f

  Copy full SHA for 5569f3f

Commits on Mar 3, 2019

• Added many comments to A3C train code to aid understanding, reshuffled things and possibly fixed big bug on not calculating final reward in total reward for episode!?!. Renamed R to curr_return and…

  Show description for b8d85a6
  beduffy

  committed
  b8d85a6

  Copy full SHA for b8d85a6

• Added warning for empty acceptable_receptacles config param and fixed if statement checks in 3 places regarding pickup put interaction. Added NL_pickup multiple bowls_vs_cups and 5 objects configs+…

  Show description for 5456517
  beduffy

  committed
  5456517

  Copy full SHA for 5456517

Commits on Mar 4, 2019

• Fixed self.controller.start() indent bug, added missing cv2 import to env_atari.py, fixed docstring about square image. Added comment about detach.

  Show description for 8e4320d
  beduffy

  committed
  8e4320d

  Copy full SHA for 8e4320d

Commits on Mar 7, 2019

• Merge branch 'master' into A3C_LSTM_GA_barebones with cups-rl name change

  beduffy

  committed
  87713bf

  Copy full SHA for 87713bf

• A3C calculate_lstm_input_size_for_A3C() doesn't need square images by using resolution[1]

  beduffy

  committed
  9c7b08f

  Copy full SHA for 9c7b08f

Commits on Mar 8, 2019

• Made README.md much better (split into subsections and gave contructor example), Added todo about seed in ai2thor, default reward for 5 objects is 5 and added to natural language tasks, removed war…

  Show description for e2cadb1
  beduffy

  committed
  e2cadb1

  Copy full SHA for e2cadb1

Commits on Mar 9, 2019

• Added EPIC 3 figures of ASCII art for A3C_LSTM_GA architecture and other comments/todos in model.py, my_optim.py and train.py

  Show description for 2001282
  beduffy

  committed
  2001282

  Copy full SHA for 2001282

• Added ActorCritic ASCII chart, cleaned up comments in model.py and made clearer. Split A3C_LSTM_GA docstring in two and explained langauge grounding. self.num_output_filters = 64. Explained GA expa…

  Show description for 5dc2fd5
  beduffy

  committed
  5dc2fd5

  Copy full SHA for 5dc2fd5

Commits on Mar 10, 2019

• Added 13 todos to main.py, fixed big bug on storing all_rewards_in_episode only in done, put 4 appends before done (why was it otherwise in the first place?) and removed 2nd "if done", fixed ai2tho…

  Show description for 396ff42
  beduffy

  committed
  396ff42

  Copy full SHA for 396ff42

Commits on Mar 14, 2019

• Put input_size functions as static methods for A3C and A3C_LSTM_GA, Added eid info to README, default reward is 1 for 5 objects

  Show description for 06f5478
  beduffy

  committed
  06f5478

  Copy full SHA for 06f5478

Commits on Mar 17, 2019

• Added show_instance_segmentation (binary mask matplotlib image on specified object types) and example to pdb_interactive py file

  beduffy

  committed
  639d9c7

  Copy full SHA for 639d9c7

• Added _process_frame42 as staticmethod in env_atari.py. In env_vizdoom code: No more import star for vizdoom. Style changes, docs and avoid repetition

  beduffy

  committed
  dd338a1

  Copy full SHA for dd338a1

Commits on Mar 20, 2019

• Removed seed init param from ai2thor_env, added lastObjectClosed, lastObjectOpened, lastObjectPickedUp, lastObjectPut, lastObjectPutReceptacle. (need to test). Added todos, time embedding explanation

  beduffy

  committed
  e3da7c9

  Copy full SHA for e3da7c9

Commits on Mar 23, 2019

• Fixed if indent on open and close objects, simplified logic into boolean vars so if statement looks easy to read, set last vars to None

  beduffy

  committed
  af95bab

  Copy full SHA for af95bab

Commits on Apr 5, 2019

• Renamed to has_language_instructions, todos, notes

  beduffy

  committed
  922d691

  Copy full SHA for 922d691

Commits on Apr 6, 2019

• Added more todos, lots of unfinished stuff (pushing for remote work purposes), writer problems, printing latest_sys_args, args.env, 3 objects config

  beduffy

  committed
  70aaf76

  Copy full SHA for 70aaf76