Skip to content

Latest commit

 

History

History
180 lines (176 loc) · 4.71 KB

134.md

File metadata and controls

180 lines (176 loc) · 4.71 KB

Results of Code LLama 34B on HumanEval/134

Date: 01 September, 2023
Model: codellama-34b-instruct
Temperature: 0.2
Passing runs: 0
Total runs: 10
Success rate: 0%

Prompt

def check_if_last_char_is_a_letter(txt):
    '''
    Create a function that returns True if the last character
    of a given string is an alphabetical character and is not
    a part of a word, and False otherwise.
    Note: "word" is a group of characters separated by space.

    Examples:
    check_if_last_char_is_a_letter("apple pie") ➞ False
    check_if_last_char_is_a_letter("apple pi e") ➞ True
    check_if_last_char_is_a_letter("apple pi e ") ➞ False
    check_if_last_char_is_a_letter("") ➞ False 
    '''

The above is an incomplete Python code fragment. Return the complete and correct code with no additional text.

Correct solution

def check_if_last_char_is_a_letter(txt):
    '''
    Create a function that returns True if the last character
    of a given string is an alphabetical character and is not
    a part of a word, and False otherwise.
    Note: "word" is a group of characters separated by space.

    Examples:
    check_if_last_char_is_a_letter("apple pie") ➞ False
    check_if_last_char_is_a_letter("apple pi e") ➞ True
    check_if_last_char_is_a_letter("apple pi e ") ➞ False
    check_if_last_char_is_a_letter("") ➞ False 
    '''check = txt.split(' ')[-1]
    return True if len(check) == 1 and (97 <= ord(check.lower()) <= 122) else False

Unit tests

def check(candidate):

    # Check some simple cases
    assert candidate("apple") == False
    assert candidate("apple pi e") == True
    assert candidate("eeeee") == False
    assert candidate("A") == True
    assert candidate("Pumpkin pie ") == False
    assert candidate("Pumpkin pie 1") == False
    assert candidate("") == False
    assert candidate("eeeee e ") == False
    assert candidate("apple pie") == False
    assert candidate("apple pi e ") == False

    # Check some edge cases that are easy to work out by hand.
    assert True

Runs

❌ Run 1

Start time: 2023-09-01 02:46:25
Duration: 4.41s
Result: Fail

def check_if_last_char_is_a_letter(txt):
    if len(txt) == 0:
        return False
    if txt[-1].isalpha() and txt[-1] not in "aeiou":
        return True
    return False

❌ Run 2

Start time: 2023-09-01 02:49:23
Duration: 4.28s
Result: Fail

def check_if_last_char_is_a_letter(txt):
    if len(txt) == 0:
        return False
    if txt[-1].isalpha() and txt[-1] not in " ":
        return True
    return False

❌ Run 3

Start time: 2023-09-01 02:52:20
Duration: 4.42s
Result: Fail

def check_if_last_char_is_a_letter(txt):
    if len(txt) == 0:
        return False
    if txt[-1].isalpha() and txt[-1] not in " ":
        return True
    return False

❌ Run 4

Start time: 2023-09-01 02:55:13
Duration: 4.99s
Result: Fail

def check_if_last_char_is_a_letter(txt):
    if len(txt) == 0:
        return False
    last_char = txt[-1]
    return last_char.isalpha() and last_char not in txt[:-1]

❌ Run 5

Start time: 2023-09-01 02:58:12
Duration: 5.21s
Result: Fail

def check_if_last_char_is_a_letter(txt):
    if len(txt) == 0:
        return False
    if txt[-1].isalpha() and txt[-1] not in " ":
        return True
    return False

❌ Run 6

Start time: 2023-09-01 03:01:17
Duration: 4.99s
Result: Fail

def check_if_last_char_is_a_letter(txt):
    if len(txt) == 0:
        return False
    if txt[-1].isalpha() and txt[-1] not in " ":
        return True
    return False

❌ Run 7

Start time: 2023-09-01 03:04:14
Duration: 4.75s
Result: Fail

def check_if_last_char_is_a_letter(txt):
    if len(txt) == 0:
        return False
    if txt[-1].isalpha() and txt[-1] not in " ":
        return True
    return False

❌ Run 8

Start time: 2023-09-01 03:07:17
Duration: 4.68s
Result: Fail

def check_if_last_char_is_a_letter(txt):
    if len(txt) == 0:
        return False
    if txt[-1].isalpha() and txt[-1] not in " ":
        return True
    return False

❌ Run 9

Start time: 2023-09-01 03:10:20
Duration: 4.42s
Result: Fail

def check_if_last_char_is_a_letter(txt):
    if len(txt) == 0:
        return False
    if txt[-1].isalpha() and txt[-1] not in " ":
        return True
    return False

❌ Run 10

Start time: 2023-09-01 03:13:18
Duration: 4.17s
Result: Fail

def check_if_last_char_is_a_letter(txt):
    if len(txt) == 0:
        return False
    if txt[-1].isalpha() and txt[-1] not in " ":
        return True
    return False