uniform_unit_scaling_initializer full_shape #6
Comments
|
@adelra Any solution you might have found? |
|
Have you tried using:
inputs = [
tf.squeeze(v, [1])
for v in tf.split(value=x, num_or_size_splits=hps.num_steps, axis=1)
]
as an attempt to fix:
File "/content/drive/app/lm-master/language_model.py", line 68, in _forward
inputs = [tf.squeeze(v, [1]) for v in tf.split(1, hps.num_steps, x)]
…On Wed, Aug 15, 2018 at 7:53 AM Nelly Mincheva ***@***.***> wrote:
@adelra <https://github.com/adelra> Any solution you might have found?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#6 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AK_u-xfrdheDwGejkajAXbV-tIDOs9oXks5uRDXPgaJpZM4TLY8n>
.
--
-Ciprian
|
|
@nellymin @adelra @ciprian-chelba @rafaljozefowicz Any success fixing this issues? I tried to change to But I still get this error |
|
@ciprian-chelba Yes the error is Sorry but I didn't understand what you are proposing or what are the links you sent? I need to enter username + password to see it |
|
@ciprian-chelba Can you please put the code? I can't go into these links |
|
Sorry, I didn't realize I was sharing hyperlinks... You don't need to be
able to follow them
…-Ciprian
Nexus 5
On Sat, Oct 13, 2018, 13:25 okoub ***@***.*** wrote:
@ciprian-chelba <https://github.com/ciprian-chelba> Can you please put
the code? I can't go into these links
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#6 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AK_u-7kuXvQSMrdPjgjN0ZylJ1IUfWVwks5ukkw0gaJpZM4TLY8n>
.
|
|
@ciprian-chelba So can you please share the code or write what I need to do? |
|
OK,
Here is the implementation for sharded_variable I am using:
def sharded_variable(name, shape,
num_shards, init_factor=1.0,
initializer=None, dtype=tf.float32): """Sharded
version of get_variable. Args: name: variable name. shape:
variable shape. num_shards: number of desired shards.
init_factor: to be used by tf.uniform_unit_scaling_initializer.
initializer: if initializer is None then use
tf.uniform_unit_scaling_initializer o/w use the one specified.
Set to something other than None only for unit tests. dtype: data
type. Returns: list of variables with name suffixed by '_' +
shard_number. """ # The final size of the sharded variable may be
larger than requested. # This should be fine for embeddings.
shard_size = int((shape[0] + num_shards - 1) / num_shards) # The
full_shape argument is no longer available in v0.12 of TensorFlow. #
We guess at this using the number of shards. if initializer is None:
init = tf <https://cs.corp.google.com/piper///depot/google3/third_party/py/lm_lstm_1Bwds/model_utils.py?l=4&ct=xref_jump_to_def&gsn=tf&rcl=216954548>.uniform_unit_scaling_initializer(
factor=init_factor * num_shards, dtype=dtype) else: init =
initializer return [ tf.get_variable( name + '_%d' % i
<https://cs.corp.google.com/piper///depot/google3/third_party/py/lm_lstm_1Bwds/model_utils.py?l=49&ct=xref_jump_to_def&gsn=i&rcl=216954548>,
[shard_size, shape[1]], initializer=init,
dtype=dtype) for i in range(num_shards) ]
…On Sun, Oct 14, 2018 at 12:21 AM okoub ***@***.***> wrote:
@ciprian-chelba <https://github.com/ciprian-chelba> So can you please
share the code or write what I need to do?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#6 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AK_u-_WMUvTwD9ArbP8UuSOI8QwiCXroks5ukuXygaJpZM4TLY8n>
.
--
-Ciprian
|
I have the same error as @adelra . Would u please post the code in a good format? The one before is not even readable. |
|
I guess copy/pasting from the browser is a bad idea. Let me try again, this
should be plain text:
def sharded_variable(name,
shape,
num_shards,
init_factor=1.0,
initializer=None,
dtype=tf.float32):
"""Sharded version of get_variable.
Args:
name: variable name.
shape: variable shape.
num_shards: number of desired shards.
init_factor: to be used by tf.uniform_unit_scaling_initializer.
initializer: if initializer is None then use
tf.uniform_unit_scaling_initializer o/w use the one specified.
Set to something other than None only for unit tests.
dtype: data type.
Returns:
list of variables with name suffixed by '_' + shard_number.
"""
# The final size of the sharded variable may be larger than requested.
# This should be fine for embeddings.
shard_size = int((shape[0] + num_shards - 1) / num_shards)
# The full_shape argument is no longer available in v0.12 of TensorFlow.
# We guess at this using the number of shards.
if initializer is None:
init = tf.uniform_unit_scaling_initializer(
factor=init_factor * num_shards, dtype=dtype)
else:
init = initializer
return [
tf.get_variable(
name + '_%d' % i, [shard_size, shape[1]],
initializer=init,
dtype=dtype) for i in range(num_shards)
]
On Mon, Oct 15, 2018 at 10:08 PM shocho3858 <[email protected]>
wrote:
… OK, Here is the implementation for sharded_variable I am using: def
sharded_variable(name, shape, num_shards, init_factor=1.0,
initializer=None, dtype=tf.float32): """Sharded version of get_variable.
Args: name: variable name. shape: variable shape. num_shards: number of
desired shards. init_factor: to be used by
tf.uniform_unit_scaling_initializer. initializer: if initializer is None
then use tf.uniform_unit_scaling_initializer o/w use the one specified. Set
to something other than None only for unit tests. dtype: data type.
Returns: list of variables with name suffixed by '*' + shard_number. """
# The final size of the sharded variable may be larger than requested. #
This should be fine for embeddings. shard_size = int((shape[0] + num_shards
- 1) / num_shards) # The full_shape argument is no longer available in
v0.12 of TensorFlow. # We guess at this using the number of shards. if
initializer is None: init = tf
https://cs.corp.google.com/piper///depot/google3/third_party/py/lm_lstm_1Bwds/model_utils.py?l=4&ct=xref_jump_to_def&gsn=tf&rcl=216954548
<https://cs.corp.google.com/piper///depot/google3/third_party/py/lm_lstm_1Bwds/model_utils.py?l=4&ct=xref_jump_to_def&gsn=tf&rcl=216954548>.uniform_unit_scaling_initializer(
factor=init_factor * num_shards, dtype=dtype) else: init = initializer
return [ tf.get_variable( name + '*%d' % i
https://cs.corp.google.com/piper///depot/google3/third_party/py/lm_lstm_1Bwds/model_utils.py?l=49&ct=xref_jump_to_def&gsn=i&rcl=216954548,
[shard_size, shape[1]], initializer=init, dtype=dtype) for i in
range(num_shards) ]
… <#m_7690075175768191023_>
On Sun, Oct 14, 2018 at 12:21 AM okoub ***@***.***> wrote: @ciprian-chelba
<https://github.com/ciprian-chelba> https://github.com/ciprian-chelba So
can you please share the code or write what I need to do? — You are
receiving this because you were mentioned. Reply to this email directly,
view it on GitHub <#6 (comment)
<#6 (comment)>>,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AK_u-_WMUvTwD9ArbP8UuSOI8QwiCXroks5ukuXygaJpZM4TLY8n
.
-- -Ciprian
I have the same error as @adelra <https://github.com/adelra> . Would u
please post the code in a good format? The one before is not even readable.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#6 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AK_u-_5RRYaVVPR3cyZm6mbg6axb6vE0ks5ulWnNgaJpZM4TLY8n>
.
--
-Ciprian
|
|
@ciprian-chelba When using your code I get the error: Any idea? |
|
Nope, sorry, something seems wrong in the split. what is the shape of x and
what is hps.num_steps?
…On Mon, Oct 22, 2018 at 3:18 AM okoub ***@***.***> wrote:
@ciprian-chelba <https://github.com/ciprian-chelba> When using your code
I get the error:
lm/run_utils.py", line 14, in run_train
model = LM(hps, "train", ps_device)
File "lm/language_model.py", line 24, in __init__
loss = self._forward(i, xs[i], ys[i], ws[i])
File "lm/language_model.py", line 72, in _forward
for v in tf.split(value=x, num_or_size_splits=hps.num_steps, axis=1)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1326, in split
axis=axis, num_split=num_or_size_splits, value=value, name=name)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 7897, in split
"Split", split_dim=axis, value=value, num_split=num_split, name=name)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3272, in create_op
op_def=op_def)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1790, in __init__
control_input_ops)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1629, in _create_c_op
raise ValueError(str(e))
ValueError: Dimension size, given by scalar input 1 must be in range [-1, 1) for 'model/model/split' (op: 'Split') with input shapes: [], [512] and with computed input tensors: input[0] = <1>.
Any idea?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#6 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AK_u-yP3PMh3xJVHmWe8fY4loQqFxJB-ks5unZuMgaJpZM4TLY8n>
.
--
-Ciprian
|
|
@ciprian-chelba I use 1-billion-word-language-modeling-benchmark. |
|
Sorry, you're going to have to debug this yourself. My code has diverged
significantly from the GitHub project, it seems.
…On Wed, Oct 24, 2018 at 12:07 AM okoub ***@***.***> wrote:
@ciprian-chelba <https://github.com/ciprian-chelba> I use
1-billion-word-language-modeling-benchmark.
I run it like that:
python3 single_lm_run.py --datadir
./1-billion-word-language-modeling-benchmark/ --logdir a.txt
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#6 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AK_u-8JnadxCICAjTXNXL6qNXKI39heaks5uoBG_gaJpZM4TLY8n>
.
--
-Ciprian
|
|
Any one solved this error? if yes, can you please share the required modification. |
|
just go to cmd and write pip install tflearn==0.3.1 |
Hi, I'm trying to run the code on Google Colab but I'm facing the following error:
I see that in the new version of tf the full_shape is no longer an argument of
uniform_unit_scaling_initializer.I tried removing the shape argument to test, but I faced another error:
Then I tried to convert num_steps into int32 which was unsuccessful.
Basically above is my unsuccessful attempt in fixing this error. What should I do about it and how can I handle shape argument in
uniform_unit_scaling_initializerThe text was updated successfully, but these errors were encountered: