What is the use of read_indexer_xxx function?

[bondijoe27]

Hi, all. From global_search.ipynb, I noticed there is

reports = read_indexer_reports(report_df, entity_df, COMMUNITY_LEVEL)
entities = read_indexer_entities(entity_df, entity_embedding_df, COMMUNITY_LEVEL)

After checking the source code of read_indexer_reports, I found these lines:

entity_df = entity_df.groupby(["title"]).agg({"community": "max"}).reset_index()
entity_df["community"] = entity_df["community"].astype(str)
filtered_community_df = entity_df["community"].drop_duplicates()

What are these codes for, especially the 'max' here? Could anyone explain that please?

I set COMMUNITY_LEVEL=2
report_df[report_df.level<=2].shape[0] = 357 and len(reports)=288. Which reports are eliminated here? Thanks🙏

(Plus, Is there any methods to visualize community? Since I don't know which community is the parent community of another. Thanks🙏)

[natoverse]

This discussion has some comments on how the community levels work: #716

[natoverse]

This discussion has some comments on visualizing the outputs: #697