2024-05-13 - KDB Project Call Meeting Minutes

[alvin-c-shih]

Date

Monday, May 13, 2024 - 9:30am ET / 2:30pm UK

// Second Monday of every month

Attendees

Fullname	Affiliation	GitHub Username
Alvin Shih	Morgan Stanley	@alvin-c-shih
Karl Moll	FINOS	@karlmoll
Aaron Davies	Morgan Stanley	@adavies42
Peter Gyorok	KX Systems	@gyorokpeter
Ross Duffy	Morgan Stanley	@rdffy
Ian O'Dwyer	KX Systems	@iodwyer
Priyanka Tripathi	Morgan Stanley	@priyanka7255

Meeting notices

• FINOS Project leads are responsible for observing the FINOS guidelines for running project meetings. Project maintainers can find additional resources in the FINOS Maintainers Cheatsheet.

• All participants in FINOS project meetings are subject to the LF Antitrust Policy, the FINOS Community Code of Conduct and all other FINOS policies.

• FINOS meetings involve participation by industry competitors, and it is the intention of FINOS and the Linux Foundation to conduct all of its activities in accordance with applicable antitrust and competition laws. It is therefore extremely important that attendees adhere to meeting agendas, and be aware of, and not participate in, any activities that are prohibited under applicable US state, federal or foreign antitrust and competition laws. Please contact [email protected] with any questions.

• FINOS project meetings may be recorded for use solely by the FINOS team for administration purposes. In very limited instances, and with explicit approval, recordings may be made more widely available.

Agenda

• (5 min) Convene, roll call, welcome new people.
• Display FINOS Antitrust Policy summary slide
• Review Meeting Notices (see above)
• (5 min) Approve previous meeting minutes.
  • Approved by acclamation.
• (25 min) Introduction to qcluster for distributing work across multiple hosts / containers.
  • Module for distributing computation using on-prem cloud.
  • Built on top of remote_utils.q.
    • cluster.remote.startq[...] - starts one worker
    • cluster.remote.startCluster[...] - starts workers on multiple hosts
    • log.setLogLeveltrace` - to see lower-level calls
    • cluster.syncRun - returns a dictionary of status
      • returns a general list of results, with q automatically collapsing lists of dictionaries into tables
    • cluster.asyncRun - requires a clusterID
      • sent to idle worker with compatible tag
      • support for multiple simultaneous async jobs in the backlog
    • cluster.async.status- status of async tasks
    • cluster.async.results - results
    • workers can be assigned "tags", typically to indicate different levels of compute / memory resource
      • dynamic tagging based on input characteristics might be an interesting feature
      • Manhattan and L.A. zip codes can be 50x bigger than others
    • can set idle timeout for (internal) cloud workers
      • https://github.com/morganstanley/treadmill
    • much of the initial work was around retry logic that the users were happy with
• (15 min) Try Sourcegraph Cody AI Coding Assistant on a variety of q tasks and see where it's useful (perhaps with some hints / prompt engineering).
  • Free version to try at home:
    • https://sourcegraph.com/cody
  • Apache 2.0 licensed allowing one to see how it works:
    • https://github.com/sourcegraph/cody
  • Custom commands allow one to remind the LLM about the quirks of q.
    • https://docs.sourcegraph.com/cody/capabilities/commands#custom-commands
  • Can be used with on-prem, open models (possibly fine-tuned for q):
    • https://sourcegraph.com/blog/local-code-completion-with-ollama-and-cody
  • Even if it doesn't understand q well enough to generate code, can try:
    • explaining code
    • generating a first pass at documentation
    • review whether code complies with coding standards
• (5 min) AOB, Q&A, Adjourn.

Join Zoom Meeting

• https://zoom.us/j/94523415326?pwd=NkJDaUtUUlZTcHVwcUJHdG0yRFJUdz09
• Meeting ID: 945 2341 5326
• Passcode: 361990
• Find your local number: https://zoom.us/u/abTtYRZ0B6

[gyorokpeter]

Péter Györök / KX

[rdffy]

Ross Duffy / Morgan Stanley

[adavies42]

Aaron Davies/Morgan Stanley

[karlmoll]

Karl Moll / FINOS

[alvin-c-shih]

Alvin Shih / Morgan Stanley

[rianoc-kx]

Rian Ó Cuinneagáin / KX

[priyanka7255]

Priyanka Tripathi / Morgan Stanley

[karlmoll]

The FINOS press release around AI Readiness SIG

https://www.finos.org/press/finos-launches-ai-readiness

[karlmoll]

"As part of the broader AI initiative, the SIG is expected to be just the first effort to emerge from exploratory conversations held across the FINOS membership, academia and innovators in the space. This community is actively nurturing contributions to advance the creation of openly governed tools, datasets and language models to accelerate financial services generative AI innovation."

Mailing List: [email protected]
Regular Meetings to be scheduled soon, and there will be members-only workshops before and during OSFF to identify non-competitive use cases