Skip to content

Commit

Permalink
[JVM] Add logic to add suggested targets (#737)
Browse files Browse the repository at this point in the history
This PR fixes the logic for jvm test-to-harness to provide references of
some methods suggestions for LLM.

Signed-off-by: Arthur Chan <[email protected]>
  • Loading branch information
arthurscchan authored Dec 2, 2024
1 parent 174a61a commit 2d71451
Show file tree
Hide file tree
Showing 2 changed files with 18 additions and 0 deletions.
13 changes: 13 additions & 0 deletions llm_toolkit/prompt_builder.py
Original file line number Diff line number Diff line change
Expand Up @@ -1381,6 +1381,15 @@ def _extract_jvm_imports(self, src: str,

return list(sorted(results)), list(sorted(others))

def _get_jvm_public_candidates(self, proj: str) -> list[str]:
"""Helper function to retrieve list of public candidates for jvm."""
method_set = set()
methods = introspector.query_introspector_jvm_all_public_candidates(proj)
for method in methods:
if "<init>" not in method['function_name']:
method_set.add(method['function_name'])
return list(method_set)

def extract_header_files(self, text):
# Include any weird macros defined that does not have any values. This
# was found empirically to be valuable.
Expand Down Expand Up @@ -1434,6 +1443,10 @@ def build(self,
prompt_text = prompt_text.replace('{TARGET_SAMPLE_HARNESS}',
harness_sample_text)

# Extract must included methods
methods = self._get_jvm_public_candidates(self.benchmark.project)
prompt_text = prompt_text.replace('{PUBLIC_METHODS}', ','.join(methods))

# Extract import list
import_list, other_import_list = self._extract_jvm_imports(
test_source_code, classes)
Expand Down
5 changes: 5 additions & 0 deletions prompts/template_xml/jvm_requirement_test_to_harness.txt
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,11 @@ Here is a list of requirements that you MUST follow.
<item>Please use {HARNESS_NAME} as the Java class name.</item>
<item>You MUST invoke the close method of any resource class objects that implements the java.lang.AutoCloseable interface in the finally block after the target method is invoked.</item>
<item>You MUST use similar approach as the examples and tests to catch exceptions.</item>
<item>
Here is a comma separated list of public project methods. You must include the invocation of at least one of them in the generated harness. Each of the method signature follows the format of <code>[Full qualified name of the class].method_name(method_arguments)</code>.
{PUBLIC_METHODS}
</item>
<item>
<item>Do not create new variables with the same names as existing variables.
WRONG:
<code>
Expand Down

0 comments on commit 2d71451

Please sign in to comment.