Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add a jobs argument #19

Open
hayd opened this issue Apr 3, 2014 · 1 comment · May be fixed by #51
Open

add a jobs argument #19

hayd opened this issue Apr 3, 2014 · 1 comment · May be fixed by #51

Comments

@hayd
Copy link
Owner

hayd commented Apr 3, 2014

autopep8 has a jobs argument:

-j n, --jobs n        number of parallel jobs; match CPU count if value is less than 1

This would be great to have, if someone wanted to pick this up. Presumably you'd deal with each file separately (i.e. where it iterates over filenames_diff).

This is less useful than autopep8's as by definition (?) we ought to be acting on a smaller set of fixes (and it should usually take less time than a full autopep8 pass).

@hayd hayd changed the title jobs argument add a jobs argument Apr 3, 2014
@hayd hayd added this to the someday milestone Apr 4, 2014
@hayd
Copy link
Owner Author

hayd commented May 23, 2014

Note: Can use multiprocessing with a manager (cleverly does all the locking for the shared object, I think this lend well to that):

d = multiprocessing.Manager().dict()
# here f updates d with filename -> diff
procs = [multiprocessing.Process(target=f, args=(file_name, d,)) for filename in filenames)]  # or pass in d as default arg to f
for p in procs: p.start()
for p in procs: p.join()  # wait until they terminate
# Can iterate/print the result from d

Obviously should turn off verbosity printing while it's doing this (can reset it back after maybe)

@hayd hayd modified the milestones: 0.9, someday May 23, 2014
@hayd hayd linked a pull request May 28, 2014 that will close this issue
@hayd hayd modified the milestones: 0.9.1, 0.9 Sep 19, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant