Cover multilevel data?

[briatte]

The ESS also offers multilevel data.

Perhaps this could find its place in the package? I can try giving it a go if you like.

[cimentadaj]

Hi @briatte

This sounds great. Of course, feel free to working this out. However, before you do so, let's discuss the possible implementation.

Some things I'm not entirely sure about.

• Can the multilevel data be scraped with the current setup of essurvey?

I'm using xml2 which is straight foward for the round/year data (it's in html, I think). However for the multilevel data it seems that it's coming from a Java based website (if I'm correct). I've done this sort of scraping with RSelenium but I'm not completely sure we would want to add such a big package as a dependency. What are your thoughts on this?

• What is the precise difference between the multilevel data and the country data? I know it includes different regions but what does the data look like? Is it like a separate data set (such as the SDDF) which simply has the unique ID for every participant and their correspoding NUTS variables? Or is it the complete dataset including the different region variables? Does this have additional variables unavailable in the other dataset?

My main concern is whether this should be a completely different step from import_country or whether it's something similar to what I've discussed in #9 in terms of doing the merging silently.

I think if we answer some of these questions, then the implementation would become feasible.

Hey! Thanks for getting involved, these ideas are really cool and we can hopefully integrate them into the package.

[briatte]

Hi @cimentadaj

I'll have to explore, being crap at Java-based scraping :) Perhaps there's a small headless browser that might just work, and could be added as a Suggests rather than Imports.

AFAIK, the multilevel data has only the higher-level variables, plus country-region identifiers to allow merging. No microdata in those files.