Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interpret FFXIV datamines for tables #13

Open
10 tasks
ZoopOTheGoop opened this issue Sep 21, 2022 · 1 comment
Open
10 tasks

Interpret FFXIV datamines for tables #13

ZoopOTheGoop opened this issue Sep 21, 2022 · 1 comment
Assignees
Labels
enhancement New feature or request priority: high This is blocking progress or a major problem that needs fixing

Comments

@ZoopOTheGoop
Copy link
Owner

ZoopOTheGoop commented Sep 21, 2022

This is a revisit of #2 with a different design idea. Instead of just using a proc macro to pull in the tables, we're going to create a few new tiny subprojects that aim to interpret datamined FFXIV .csv files. These packages are not meant to pull any of the tables (that's easy enough with just a csv parser anyway if desired), but rather simply encoding the info relevant to us.

This will make hand-copying values much simpler. Originally #2 was marked as "not planned" because it seemed like a lot of work to save a little time, but it became clear during the course of #8 that once recipes get involved, a lot of manual reading and copying of data from the recipes and items files in particular was necessary, and very annoying to do. Faced with the prospect of upgrading to 6.2, I would prefer to just get this out of the way in order to make the process more painless in the future.

The following steps must be completed for this to be considered done:

  • Data structs for each of the major things we need (crafting lookups, recipes, etc)
    • Crafting can be loaded as either individual tables (as it is now), or blocks of structs (i.e. organized by rlvl)
    • Recipe data should include info not just in the Recipe.csv, and as needed pull data from tables like Items
    • Support for loading single records, or small batches of records, as needed
  • A proc macro project to generate this data at compile-time
    • Works on data files given via environment variables
    • Like the runtime section, generates a block of structs, or individual tables
    • For recipes, ability to only generate certain listed recipes
  • A crate that re-exports the types from the first crate, and also (via feature gate) includes the tables
  • Refactor current simulator code to pull in data from this crate

This may require moving some types from the simulator crate to the crafting data crate. We should try not moving code that's too complex or relies on the current proc macros (such as the Condition family), likely meaning any types we move will be the more raw unexposed types. The current more complex types should have a relatively trivial From or TryFrom from the new types.

@ZoopOTheGoop ZoopOTheGoop added enhancement New feature or request priority: high This is blocking progress or a major problem that needs fixing labels Sep 21, 2022
@ZoopOTheGoop ZoopOTheGoop added this to the Fully working simulator milestone Sep 21, 2022
@ZoopOTheGoop ZoopOTheGoop self-assigned this Sep 21, 2022
@ZoopOTheGoop
Copy link
Owner Author

Special note: it is very important we don't actually distribute the FFXIV data. Obviously we naturally have some FFXIV data just by having ability names and stuff like their CP costs and efficiency values etc, but in particular we cannot include the item or recipe table data, and especially not distribute that data on crates.io. Thus even the "codegen static tables" variant must not actually include the data directly.

In fact, this will decrease the amount of data directly from FFXIV in the codebase by a considerable degree.

In the future we should consider how we want to handle FFXIV data for general legal reasons (i.e. the codegen one will bake it into the executable). Obviously any live version of this tool will have to get recipe and rlvl data from somewhere, but by abstracting this out we could even make this data exclusively be load-on-demand from xivapi or similar, rather than having the file on a server (or similar).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request priority: high This is blocking progress or a major problem that needs fixing
Development

No branches or pull requests

1 participant