Multiple loops logic and speed optimization in Python? -
here 2 python functions transfer data 1 file file. both source file , target file have same number of objects different data.
def getblock(rigobj, objname): rigobj.seek(0) tag = false block = "" line in rigobj: if line.find("objectalias " + str(objname) + "\n") != -1: line in rigobj: if line.find("beginkeyframe") != -1: tag = true elif line.lstrip().startswith("0.000 ") , line.rstrip().endswith("default"): tag = false break elif tag: block += line return (block) def buildscene(sceneobj, rigobj, objlist): sceneobj.seek(0) rigobj.seek(0) newscene = "" line in sceneobj: newscene += line obj in objlist: if line.find("objectalias " + obj + "\n") != -1: tag = true line in sceneobj: if line.find("beginkeyframe") != -1: newscene += line newscene += getblock(rigobj, obj) tag = false elif line.lstrip().startswith("0.000 ") , line.rstrip().endswith("default"): newscene += line tag = true break elif tag: newscene += line return (newscene)
getblock sub-function getting data rigobj;
buildscene main function, has 3 parameters:
first parameter(sceneobj) file want put data into;
second parameter(rigobj) file data from;
third parameter(objlist) list of object's data transfered.
so far, function job, problem bit of slow(sceneobj<10mb, rigobj<2mb, objlist<10 objects), not sure if there logic problem in code, should loop sceneobj first or loop objlist first? affect speed?
update:
both sceneobj , rigobj have similar data this:
lines beginobject lines objectalias xxx #--> object in transfer list lines beginkeyframe 10 12 -9.000 4095 default #--> transfer begins lines #--> transfer rigobj sceneobj , override lines in sceneobj -8.000 63 default #--> same lines #--> same -7.000 63 default #--> same lines #--> same -1.000 63 default #--> same lines #--> transfer ends 0.000 -1 default lines endkeyframe endmotion lines endobject
the data want transfered , overrided lines bewteen beginkeyframe , 0.000 -1 default of specified objects(by objlist)
most obvious optimization index data getblock
function, able seek needed position instead of parsing full file beginning.
like so:
def create_rig_index(rig_obj): """ function creates dict of offsets specific objectalias example: data: line offset 100: objectalias xxx more lines line offset 200: objectalias yyy more lines line offset 300: objectalias xxx more lines result be: xxx: [100, 300] yyy: [200] """ idx = defaultdict( list ) position = 0 line in rig_obj: strip_line = line.strip() if strip_line.startswith( "objectalias" ): obj_name = strip_line.split()[1] idx[ obj_name ].append( position ) # unfortunately python prevent `tell` calls during iteration. position += len( bytes( line, 'utf-8' ) ) # if data guaranteed ascii possible use len( line ) # or can write custom line generator on top of read function. return idx; def getblock(rigobj, rigidx, objname): """ same getblock, uses precalculated offsets""" block = "" idx in rigidx[ objname ]: rigobj.seek( idx ) tag = false line in rigobj: if line.find("beginkeyframe") != -1: tag = true elif line.lstrip().startswith("0.000 ") , line.rstrip().endswith("default"): break elif tag: block += line return (block)
in buildscene method should create rig_index before running loop, , use index in getblock
function.
Comments
Post a Comment