Code

update readme to AWS#1

Code

Open

yinshiyi wants to merge 45 commits intor1.6.1yinshiyi/deepvariant:r1.6.1from AWS_updateyinshiyi/deepvariant:AWS_updateCopy head branch name to clipboard

+869-113

Commits

Commits on Jun 21, 2024

• first update

  yinshiyi

  committed
  7ce9ea7

  Copy full SHA for 7ce9ea7

Commits on Sep 13, 2024

• plan update

  yinshiyi

  committed
  c05c778

  Copy full SHA for c05c778

Commits on Sep 17, 2024

• add coverage script and make sure happy is run using coverage >2

  yinshiyi

  committed
  53d9a9e

  Copy full SHA for 53d9a9e

Commits on Sep 26, 2024

• launch small instance

  yinshiyi

  committed
  11f4bc3

  Copy full SHA for 11f4bc3

Commits on Oct 2, 2024

• connect aws ec2

  yinshiyi

  committed
  919c2c1

  Copy full SHA for 919c2c1

Commits on Oct 3, 2024

• oct 2 2024 progress

  yinshiyi

  committed
  9f14838

  Copy full SHA for 9f14838

• oct 3 2024 progress

  yinshiyi

  committed
  adf2e41

  Copy full SHA for adf2e41

Commits on Oct 4, 2024

• oct 4 2024

  yinshiyi

  committed
  b253057

  Copy full SHA for b253057

• finish making example

  yinshiyi

  committed
  a43f19f

  Copy full SHA for a43f19f

• add the transfer script

  yinshiyi

  committed
  9c1bb65

  Copy full SHA for 9c1bb65

Commits on Oct 7, 2024

• add shuffle path

  yinshiyi

  committed
  85bf27c

  Copy full SHA for 85bf27c

Commits on Oct 16, 2024

• add worker and update dependency

  yinshiyi

  committed
  b4f8941

  Copy full SHA for b4f8941

• add upload script

  yinshiyi

  committed
  ddaf0a3

  Copy full SHA for ddaf0a3

Commits on Oct 17, 2024

• add validation data

  yinshiyi

  committed
  f223138

  Copy full SHA for f223138

• add shuffle validation

  yinshiyi

  committed
  2c99126

  Copy full SHA for 2c99126

• add more cpu thread

  yinshiyi

  committed
  db46bd8

  Copy full SHA for db46bd8

• update shuffle code

  yinshiyi

  committed
  07899f6

  Copy full SHA for 07899f6

• testing shuffle

  yinshiyi

  committed
  2d9f0cf

  Copy full SHA for 2d9f0cf

Commits on Oct 18, 2024

• update file path

  yinshiyi

  committed
  f30ad08

  Copy full SHA for f30ad08

Commits on Oct 23, 2024

• update dateflow to spark

  yinshiyi

  committed
  5dc1599

  Copy full SHA for 5dc1599

• add training command

  yinshiyi

  committed
  05c0db2

  Copy full SHA for 05c0db2

• fix file path

  yinshiyi

  committed
  e455e56

  Copy full SHA for e455e56

• add final eval

  yinshiyi

  committed
  311c6d4

  Copy full SHA for 311c6d4

Commits on Oct 24, 2024

• add baseline and hap

  yinshiyi

  committed
  9e5bea8

  Copy full SHA for 9e5bea8

Commits on Nov 7, 2024

• typo fix

  yinshiyi

  committed
  3d91f9a

  Copy full SHA for 3d91f9a

• update flags

  yinshiyi

  committed
  cd6276d

  Copy full SHA for cd6276d

Commits on Nov 23, 2024

• Update shuffle_tfrecords_beam.py

  yinshiyi

  authored
  84363f6

  Copy full SHA for 84363f6

Commits on Dec 4, 2024

• add the real data to try

  yinshiyi

  committed
  7efb829

  Copy full SHA for 7efb829

Commits on Dec 5, 2024

• Update shuffle_tfrecords_beam.py

  yinshiyi

  authored
  7a54cce

  Copy full SHA for 7a54cce

• Update shuffle_tfrecords_beam.py

  yinshiyi

  authored
  8a55673

  Copy full SHA for 8a55673

• Create beam_test.py

  yinshiyi

  authored
  85b58dd

  Copy full SHA for 85b58dd

• add test files

  yinshiyi

  committed
  92a8c28

  Copy full SHA for 92a8c28

• fix typo and update subversion to contain aws and boto3

  yinshiyi

  committed
  8fb4a2d

  Copy full SHA for 8fb4a2d

• create cluster programatically

  yinshiyi

  committed
  f641344

  Copy full SHA for f641344

• beam spark local test file

  yinshiyi

  committed
  25edaa7

  Copy full SHA for 25edaa7

Commits on Dec 6, 2024

• emr cluster

  yinshiyi

  committed
  0f9f527

  Copy full SHA for 0f9f527

• docker file

  yinshiyi

  committed
  fa757ac

  Copy full SHA for fa757ac

Commits on Dec 10, 2024

• Create test2024.sh

  yinshiyi

  authored
  12ed0b9

  Copy full SHA for 12ed0b9

• Create beam_test_2024.py

  yinshiyi

  authored
  6b3fa4b

  Copy full SHA for 6b3fa4b

• Update beam_test_2024.py

  yinshiyi

  authored
  8a8c99d

  Copy full SHA for 8a8c99d

• Update beam_test_2024.py

  yinshiyi

  authored
  ca85bfe

  Copy full SHA for ca85bfe

• Update beam_test_2024.py

  yinshiyi

  authored
  abb57fe

  Copy full SHA for abb57fe

Commits on Dec 13, 2024

• update notes

  yinshiyi

  committed
  7b719ea

  Copy full SHA for 7b719ea

• to do list

  yinshiyi

  committed
  b27a451

  Copy full SHA for b27a451

Commits on Jan 18, 2025

• manually input the yarn address, future to dynamically capture the address for each new cluster

  yinshiyi

  committed
  be7b6fd

  Copy full SHA for be7b6fd