Serializing large Usd files into maya ASCII crashes maya

[alekamca]

We probably shouldn't be storing our data like this, so that's a workflow issue on our end, but I thought I would submit an issue here.

We're generating fairly large edit layers (~10MB of text), and we're running into issues where Maya (currently Maya 2018 Update 3) is crashing when trying to save these files (in batch mode). We've tracked down the issue being the LayerManager trying set the serialized layer into one giant MString attribute. We've run into this bug in Maya in the past, so it does seem to be at its core a Maya problem, not a USDMaya problem.

[alekamca]

Python script to reproduce at least in our environment (Maya 2018 Update3):

# from within maya batch mode
from maya import cmds as mc
from pxr import Usd, Vt, Sdf

print 'creating usd stage'
# create usd stage with a large ascii representation
stage = Usd.Stage.CreateNew('tmpStage.usda')
prim = stage.DefinePrim('/prim')
attr = prim.CreateAttribute('doubleArray', Sdf.ValueTypeNames.DoubleArray)
arrayValues = Vt.DoubleArray(1000000, 1.0/3.0)
attr.Set(arrayValues)

# import stage
print 'importing stage'
mc.AL_usdmaya_ProxyShapeImport(f='tmpStage.usda')

# save file
print 'saving'
mc.file(rename='test.ma')
mc.file(save=True,force=True,type="mayaAscii") #crash occurs here
print 'done'

[dbaz]

Havn't thought about it too deeply, but you could update the LayerManagerNode to have the option of saving out a sidecar usda file that contains the contents instead of having it inline. The contents could change to being the filepath to this sidecar file and the identifier kept the same.

At this stage I don't think we would use it, we have already had problems with sidecar files where artists would move around their Maya file, or just delete the sidecar files, and essentially breaking their Maya file or losing data.

[murphyeoin]

We're going to look at this one soon - it's important to get this working

[robthebloke]

I'm taking a look at using a custom MPxData attribute type to work around the hard limit. Ideally when in the land of *.mb files, I'd prefer to serialise out as a usdc binary blob, however there doesn't seem to be a way to do this in USD currently. Question asked here....

https://groups.google.com/forum/#!topic/usd-interest/_WVE4tHojdc

I'll give it a go with strings, and put a PR up for you to test....