Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Process positional arguments #134

Open
wants to merge 61 commits into
base: main
from
Open
Changes from 6 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
df69fd5
parsing positional arguments for tf.function decorator
tatianacv Dec 12, 2022
3b881a6
fixing comments
tatianacv Dec 12, 2022
2899de3
adding tests
tatianacv Dec 12, 2022
731b664
fix trailing whitespace
tatianacv Dec 12, 2022
a4cf2cd
fix
tatianacv Dec 12, 2022
d6b16ed
not counting the default values as existing
tatianacv Dec 13, 2022
aeca489
fix, might need to tidy up
tatianacv Dec 13, 2022
61f1b1b
fix
tatianacv Dec 13, 2022
e8b508c
fetching from upstream
tatianacv Dec 14, 2022
109e0e4
Progress
tatianacv Dec 14, 2022
b7d9d5c
fix trailing whitespace
tatianacv Dec 14, 2022
6dbbadd
Throwing exceptions
tatianacv Dec 15, 2022
e5c5b80
adding tests for another tf version
tatianacv Dec 16, 2022
1046cff
changing test's tf.function version and some formatting
tatianacv Dec 17, 2022
8c489aa
fixing python code
tatianacv Dec 17, 2022
130f807
fixing to a non-default value
tatianacv Dec 17, 2022
55cadea
update
tatianacv Dec 17, 2022
c83eccf
adding requirement
tatianacv Dec 17, 2022
4361e2e
update
tatianacv Dec 17, 2022
53c44f1
fix checkstyle
tatianacv Dec 17, 2022
23ae949
update
tatianacv Jan 3, 2023
94a005f
fixing comments
tatianacv Jan 9, 2023
528a3a7
Merge branch 'ponder-lab:main' into issue_108
tatianacv Jan 9, 2023
a3cfae6
progress on comments
tatianacv Jan 9, 2023
92cf918
Merge branch 'issue_108' of https://github.com/tatianacv/Hybridize-Fu…
tatianacv Jan 9, 2023
b145a53
progress
tatianacv Jan 9, 2023
72d53c6
removing code that has to do with keywords, not positionals
tatianacv Jan 10, 2023
d05963b
Adding newline
tatianacv Jan 10, 2023
fffd0d0
removing code that accounts for default values, adding documentation,…
tatianacv Jan 13, 2023
49065c9
removing a character from a comment
tatianacv Jan 13, 2023
f53fe07
Merge branch 'main' into issue_108
khatchad Jan 18, 2023
e52d371
changing the versions to constants
tatianacv Jan 19, 2023
ca0c4d2
add comments regarding the declaring defintions
tatianacv Jan 19, 2023
914c417
Merge branch 'issue_108' of https://github.com/tatianacv/Hybridize-Fu…
tatianacv Jan 19, 2023
ebd3880
changing variable name
tatianacv Jan 19, 2023
aa4f6cd
Merge branch 'main' into issue_108
khatchad Jan 23, 2023
19a8a72
fixing issue #, changing order of assert clause
tatianacv Jan 23, 2023
f92872c
Merge branch 'issue_108' of https://github.com/tatianacv/Hybridize-Fu…
tatianacv Jan 23, 2023
1eea97a
Merge branch 'main' into issue_108
khatchad Jan 24, 2023
f7cd804
Update
tatianacv Jan 26, 2023
a107678
Merge branch 'main' into issue_108
tatianacv Jan 31, 2023
a835e8b
Pass build
tatianacv Feb 24, 2023
594ea9d
Adding more comments
tatianacv Feb 24, 2023
dbf19ad
Merge branch 'main' into issue_108
tatianacv Mar 4, 2023
c182a37
Updates on requested changes
tatianacv Mar 6, 2023
d3239b4
update
tatianacv Mar 10, 2023
9a60cbf
update
tatianacv Mar 10, 2023
1cb7502
Renaming for clarity
tatianacv Mar 10, 2023
f710611
update
tatianacv Mar 10, 2023
dacd17a
Merge branch 'main' into issue_108
khatchad Mar 16, 2023
5673690
update
tatianacv Mar 17, 2023
474acdf
Merge branch 'issue_108' of https://github.com/tatianacv/Hybridize-Fu…
tatianacv Mar 17, 2023
ccce776
Merge branch 'main' into issue_108
khatchad Mar 20, 2023
4e16587
Merge branch 'main' into issue_108
tatianacv Mar 21, 2023
5d72476
Changing comments, removing keyword args else
tatianacv Mar 22, 2023
b4358fd
Merge branch 'issue_108' of https://github.com/tatianacv/Hybridize-Fu…
tatianacv Mar 22, 2023
54ba275
Checking that the size of the parse tffunction args is less than defi…
tatianacv Mar 22, 2023
afd01ff
Adding more info
tatianacv Mar 23, 2023
1ca2997
Merge branch 'main' into issue_108
tatianacv Mar 23, 2023
afdbbb0
Merge branch 'main' into issue_108
khatchad Mar 28, 2023
4161ad7
Merge branch 'main' into issue_108
tatianacv Mar 31, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ public HybridizationParameters(IProgressMonitor monitor) throws BadLocationExcep
decoratorsType tfFunctionDecorator = null;

// Declaring definitions of the decorator, if it contains multiple definitions there might be more than one in this set. Since
// we are dealing with tf.function, we expect only one.
// we are dealing with tf.function, we check we it has one definition.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not only the comment. The checking must be in the code.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a check in the code.

Definition declaringDefinition = null;

// Iterate through the decorators of the function
Expand All @@ -164,61 +164,62 @@ public HybridizationParameters(IProgressMonitor monitor) throws BadLocationExcep
if (potentialDeclaringDefitinion.iterator().hasNext())
declaringDefinition = potentialDeclaringDefitinion.iterator().next();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is there not an else case? I'm not seeing the logic here. Can be greatly helped with some comments in the code of the case analysis.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added comments for this conditional.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know what variable would be null and why. Rewrite the logic to handle the else case here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have rewritten so we handle the else case here

else
throw new IllegalStateException("Can't determine tf.function decorator defintion.");
throw new IllegalStateException(String.format(
"Can't find declaring definition for selection: %s in line: %s, file: %s, and project: %s.",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This error message is vague and doesn't resemble the one that was modified.

selection.getSelectedText(), selection.getLineWithoutCommentsOrLiterals().strip(),
Function.this.containingFile.getName(), Function.this.nature.getProject()));
}
} catch (AmbiguousDeclaringModuleException e) {
throw new IllegalStateException("Can't determine whether decorator: " + decorator + " is hybrid.", e);
}
} // We expect to have the last tf.function decorator in tfFunctionDecorator

// Getting tf.functions Python definition arguments.
ArrayList<String> argumentIdDeclaringDefintion = getTfFunctionPythonDefinitionArguments(declaringDefinition);
ArrayList<String> argumentIdDeclaringDefintion = getPythonDefinitionArguments(declaringDefinition);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is a "Python definition argument?"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting the argument from the Python definition


if (tfFunctionDecorator != null)
// tfFunctionDecorator must be an instance of Call, because that's the only way we have parameters.
if (tfFunctionDecorator.func instanceof Call) {
Call callFunction = (Call) tfFunctionDecorator.func;

exprType[] tfFunctionPositionalArgs = callFunction.args;

// We iterate over the tf.function's parameters positions.
for (int i = 0; i < callFunction.args.length; i++) {
// From the position i, we use the tf.function's definition to verify which parameter we are analyzing.
String evaluatedArgument = argumentIdDeclaringDefintion.get(i);

// Matching the arguments from the definition and the arguments from the code being analyzed.
if (evaluatedArgument.equals(FUNC))
// Found parameter func
this.funcParamExists = true;
else if (evaluatedArgument.equals(INPUT_SIGNATURE))
// Found parameter input_signature
this.inputSignatureParamExists = true;
else if (evaluatedArgument.equals(AUTOGRAPH))
// Found parameter autograph
this.autoGraphParamExists = true;
// In our accepted interval version ([2.0,2.11]) of the API allows parameter names jit_compile and
// deprecated name experimental_compile.
else if (evaluatedArgument.equals(JIT_COMPILE) || evaluatedArgument.equals(EXPERIMENTAL_COMPILE))
// Found parameter jit_compile/experimental_compile
this.jitCompileParamExists = true;
// In our accepted interval version ([2.0,2.11]) of the API allows parameter names reduce_retracing
// and deprecated name experimental_relax_shapes.
else if (evaluatedArgument.equals(REDUCE_RETRACING))
// Found parameter reduce_retracing
this.reduceRetracingParamExists = true;
else if (evaluatedArgument.equals(EXPERIMENTAL_RELAX_SHAPES))
// Found parameter experimental_relax_shapes
this.reduceRetracingParamExists = true;
else if (evaluatedArgument.equals(EXPERIMENTAL_IMPLEMENTS))
// Found parameter experimental_implements
this.experimentalImplementsParamExists = true;
else if (evaluatedArgument.equals(EXPERIMENTAL_AUTOGRAPH_OPTIONS))
// Found parameter experimental_autograph_options
this.experimentalAutographOptionsParamExists = true;
else if (evaluatedArgument.equals(EXPERIMENTAL_FOLLOW_TYPE_HINTS))
// Found parameter experimental_follow_type_hints
this.experimentaFollowTypeHintsParamExists = true;
else
throw new IllegalArgumentException("Unable to process tf.function argument.");
}
if (tfFunctionPositionalArgs.length <= argumentIdDeclaringDefintion.size()) {
for (int i = 0; i < tfFunctionPositionalArgs.length; i++) {
// From the position i, we use the tf.function's definition to verify which parameter we are analyzing.
String evaluatedArgument = argumentIdDeclaringDefintion.get(i);

// Matching the arguments from the definition and the arguments from the code being analyzed.
if (evaluatedArgument.equals(FUNC))
this.funcParamExists = true;
else if (evaluatedArgument.equals(INPUT_SIGNATURE))
this.inputSignatureParamExists = true;
else if (evaluatedArgument.equals(AUTOGRAPH))
this.autoGraphParamExists = true;
// In our accepted interval version ([2.0,2.11]) of the API allows parameter names jit_compile and
// deprecated name experimental_compile.
else if (evaluatedArgument.equals(JIT_COMPILE) || evaluatedArgument.equals(EXPERIMENTAL_COMPILE))
this.jitCompileParamExists = true;
// In our accepted interval version ([2.0,2.11]) of the API allows parameter names reduce_retracing
// and deprecated name experimental_relax_shapes.
else if (evaluatedArgument.equals(REDUCE_RETRACING))
this.reduceRetracingParamExists = true;
else if (evaluatedArgument.equals(EXPERIMENTAL_RELAX_SHAPES))
this.reduceRetracingParamExists = true;
else if (evaluatedArgument.equals(EXPERIMENTAL_IMPLEMENTS))
this.experimentalImplementsParamExists = true;
else if (evaluatedArgument.equals(EXPERIMENTAL_AUTOGRAPH_OPTIONS))
this.experimentalAutographOptionsParamExists = true;
else if (evaluatedArgument.equals(EXPERIMENTAL_FOLLOW_TYPE_HINTS))
this.experimentaFollowTypeHintsParamExists = true;
else
throw new IllegalArgumentException("Unable to process tf.function argument at position " + i);
}
} else
throw new IllegalArgumentException(
"Unable to process tf.function argument. The number of arguments " + tfFunctionPositionalArgs.length
+ " exceeds the accepted number of argumets " + argumentIdDeclaringDefintion.size());
tatianacv marked this conversation as resolved.
Show resolved Hide resolved

// Processing keywords arguments
// If we have keyword parameter, afterwards, we cannot have positional parameters because it would result in invalid
Expand Down Expand Up @@ -256,20 +257,18 @@ else if (name.id.equals(EXPERIMENTAL_AUTOGRAPH_OPTIONS))
else if (name.id.equals(EXPERIMENTAL_FOLLOW_TYPE_HINTS))
// Found parameter experimental_follow_type_hints
this.experimentaFollowTypeHintsParamExists = true;
else
throw new IllegalArgumentException("Unable to process tf.function argument.");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why was this removed? Now, we are missing the else case here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this is for keyword arguments

}
}
} // else, tf.function is used without parameters.
}

/**
* Get the tf.function parameter names from the {@link Definition}.
* Get the parameter names from the {@link Definition}.
*
* @param declaringDefinition The Definition to use.
* @return An array with the names of the arguments given by {@link Definition}.
*/
private ArrayList<String> getTfFunctionPythonDefinitionArguments(Definition declaringDefinition) {
private ArrayList<String> getPythonDefinitionArguments(Definition declaringDefinition) {
// Python source arguments from the declaring definition
exprType[] declaringArguments = null;

Expand Down