14

I just completed writing my first program in python, i have written all my functions in a single module, i just executed it from command line by giving the input files as argument and it worked. But when i gave a big dataset, my program continuously running for some time. Now my next step is to find which function is taking more time in my module. I could get the time taken by whole program, but i need for each function separately.

I tried to understand timeit and profile modules in python, but as per my understanding, they were giving the time taken by a particular function. Is there a way to know the time taken by each function in my module as a statistics( all at once )?

Thanks in advance.

Improve this question
1
  • 2
    Try this. Commented Sep 21, 2014 at 12:00

2 Answers 2

Reset to default
22

At the terminal, run

python -m profile -s time file.py

or

python -m cProfile -s time file.py

The second can be faster, and is never worse.

This will give something like:

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
       39    0.132    0.003    0.139    0.004 :0(load_dynamic)
      239    0.097    0.000    0.097    0.000 :0(loads)
    541/1    0.083    0.000    3.556    3.556 :0(exec)
       30    0.082    0.003    0.082    0.003 :0(statusBar)
                        ... etc ...

The left hand side will contain your functions.

Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

Am getting these errors --> File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main "main", mod_spec) File "/usr/lib/python3.4/runpy.py", line 85, in _run_code exec(code, run_globals) File "/usr/lib/python3.4/profile.py", line 589, in <module> main() File "/usr/lib/python3.4/profile.py", line 575, in main code = compile(fp.read(), progname, 'exec') File "/home/****/eclipseworkspace/abcd.py", line 20 print '%r (%r, %r) %2.2f sec' % \
What happens if you just run python file.py?
Also, what does python -V say? It should start with "Python 2".
I have both 2.7 and 3.4.0 as part of my ubuntu 14 version. And so i changed the default version to Python 3.4.0
Right, so you'll need to run python2 -m profile -s time file.py as your code is Python 2. print '%r (%r, %r) %2.2f sec' is Python 2 syntax.
|
6

first i suggest use profilers module or timeit for this aim .timeit provides a simple way to time small bits of Python code !

To profile a function that takes a single argument, you can do:

import cProfile
import re
cProfile.run('re.compile("foo|bar")')

Also you can use a Decorator like this that allows you to measure the execution times of dedicated methods :

import time                                                

def measure_time(f):

  def timed(*args, **kw):
    ts = time.time()
    result = f(*args, **kw)
    te = time.time()

    print '%r (%r, %r) %2.2f sec' % \
          (f.__name__, args, kw, te-ts)
    return result

return timed

You can use it like this :

  @measure_time
  def foo():
        #content of function

note that f.__name__ return the name of function ! (in this case 'foo')

Improve this answer

8 Comments

Sorry, i have never done this before. So, i have to add this measure_time function in the very starting of my program and add @measure_time before every function of mine? Am i right?
Is it possible to use profile or timeit for the whole program? And does that give me time taken by every function or i have to use them to find the time taken by one function once?
you need define the function in your .py and use @measure_time be fore any function for find those runtime !
its best to use profile and you can use it for all of your code
To profile a function that takes a single argument, you can do: import cProfile import re cProfile.run('re.compile("foo|bar")')
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.