[SPARK-50685][PYTHON][FOLLOW-UP] Improve Py4J performance by leveraging getattr

[HyukjinKwon]

What changes were proposed in this pull request?

This PR is. a followup of #49313 that fixes more places missed.

This PR fixes Core, SQL, ML and Structured Streaming. Tests codes, MLLib and DStream are not affected.

Why are the changes needed?

To reduce the overhead of Py4J calls.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Manually tested as demonstrated in #49312

Was this patch authored or co-authored using generative AI tooling?

No.

[zhengruifeng]

To further optimize Py4J calls, does it make sense to cache the result? e.g.

@functools.lru_cache(maxsize=128)
def get_jvm_attr(jvm: "JVMView", name: str) -> Any:
    return getattr(jvm, name)

[HyukjinKwon]

Yeah but we should think about how it will affect GC in Python and JVM.

[HyukjinKwon]

Merged to master.