c++ - Embedded extended python multiprocessing imports -
i have c/c++ application embedding python3 interpreter , extending "emb" module outlined here https://docs.python.org/3/extending/embedding.html.
here rough example of c++
py_setpythonhome(ppath); py_setpath(path); pyimport_appendinittab("emb", &pyinit_emb); py_setprogramname(program); /* optional recommended */ py_initialize(); pysys_setargvex(argc, argv_copy, 0); pyobject *obj = py_buildvalue("s", "mypython.py"); file *file = _py_fopen_obj(obj, "r+"); if(file != null) { pyrun_simplefile(file, "mypython.py"); } py_finalize();
the python script (mypython.py) c++ app loads , works great:
import emb print("number of arguments", emb.numargs())
i modified python script (mypython.py) test multiprocessing working , works great after setting executable.
from multiprocessing import process, set_executable set_executable('c:\path\to\python3\python.exe') def worker(num): print('worker:', num) if __name__ == '__main__': jobs = [] in range(5): p = process(target=worker, args=(i,)) jobs.append(p) p.start()
lastly actual issue i'm @ now, combined both code 1 emb stuff @ top.
mypython.py
import emb print("number of arguments", emb.numargs()) multiprocessing import process, set_executable set_executable('c:\path\to\python3\python.exe') def worker(num): print('worker:', num) if __name__ == '__main__': jobs = [] in range(5): p = process(target=worker, args=(i,)) jobs.append(p) p.start()
now when code gets run first print out of number of arguments bunch of errors. can tell there problems copying modules on newly spawned python processes because coming c++ statically? i'm not sure how can fix can tell spawned processes emb import in other way? not script on python path in c++ code. need run pyimport_appendinittab
on child processes or something?
here output:
number of arguments 1 traceback (most recent call last): file "<string>", line 1, in <module> file "multiprocessing\spawn.py", line 106, in spawn_main file "multiprocessing\spawn.py", line 115, in _main file "multiprocessing\spawn.py", line 226, in prepare file "multiprocessing\spawn.py", line 278, in _fixup_main_from_path file "runpy.py", line 254, in run_path file "runpy.py", line 96, in _run_module_code file "runpy.py", line 85, in _run_code file "c:\_dev\projects\pythontest\mypython.py", line 1, in <module> traceback (most recent call last): file "<string>", line 1, in <module> import emb importerror: no module named 'emb' file "multiprocessing\spawn.py", line 106, in spawn_main file "multiprocessing\spawn.py", line 115, in _main traceback (most recent call last): file "<string>", line 1, in <module> file "multiprocessing\spawn.py", line 226, in prepare file "multiprocessing\spawn.py", line 106, in spawn_main file "multiprocessing\spawn.py", line 278, in _fixup_main_from_path file "multiprocessing\spawn.py", line 115, in _main file "multiprocessing\spawn.py", line 226, in prepare file "runpy.py", line 254, in run_path traceback (most recent call last): file "<string>", line 1, in <module> file "multiprocessing\spawn.py", line 278, in _fixup_main_from_path file "runpy.py", line 96, in _run_module_code file "multiprocessing\spawn.py", line 106, in spawn_main file "runpy.py", line 85, in _run_code file "runpy.py", line 254, in run_path file "multiprocessing\spawn.py", line 115, in _main file "c:\_dev\projects\pythontest\mypython.py", line 1, in <module> file "runpy.py", line 96, in _run_module_code file "multiprocessing\spawn.py", line 226, in prepare import emb file "runpy.py", line 85, in _run_code importerror: no module named 'emb' file "multiprocessing\spawn.py", line 278, in _fixup_main_from_path file "c:\_dev\projects\pythontest\mypython.py", line 1, in <module> file "runpy.py", line 254, in run_path import emb importerror: no module named 'emb' file "runpy.py", line 96, in _run_module_code file "runpy.py", line 85, in _run_code file "c:\_dev\projects\pythontest\mypython.py", line 1, in <module> import emb importerror: no module named 'emb' traceback (most recent call last): file "<string>", line 1, in <module> file "multiprocessing\spawn.py", line 106, in spawn_main file "multiprocessing\spawn.py", line 115, in _main file "multiprocessing\spawn.py", line 226, in prepare file "multiprocessing\spawn.py", line 278, in _fixup_main_from_path file "runpy.py", line 254, in run_path file "runpy.py", line 96, in _run_module_code file "runpy.py", line 85, in _run_code file "c:\_dev\projects\pythontest\mypython.py", line 1, in <module> import emb importerror: no module named 'emb'
Comments
Post a Comment