Tag Archives: Python

Get required lasts lines from a huge text file in python

During my work, I need to parse the result from some huge log files. The result is always save at the end of the file. So I need find a way to get the required last lines to accelerate the parse speed. Here is the function I have written to achieve this requirement in python.

  1. def last_lines(filename, lines=1):
  2. '''
  3. return the last lines from the specify files
  4. '''
  5. with open(filename, 'rb') as fh:
  6. fh.seek(0, os.SEEK_END)
  7. if fh.tell() == 0:
  8. return []
  9. fh.seek(-2, os.SEEK_END)
  10. #while not BOF
  11. while fh.tell() > 0:
  12. cur_char = fh.read(1).decode()
  13. if cur_char == '\n':
  14. lines -= 1
  15. if lines > 0:
  16. fh.seek(-2, os.SEEK_CUR)
  17. else:
  18. break
  19. return [line.decode() for line in fh.readlines()]

 

Use Python puka client with RabbitMQ

Send a message to RabbitMQ

  1. #!/usr/bin/env python
  2.  
  3. import sys
  4. sys.path.append("..")
  5.  
  6.  
  7. import puka
  8.  
  9. client = puka.Client("amqp://localhost/")
  10.  
  11. promise = client.connect()
  12. client.wait(promise)
  13.  
  14. promise = client.queue_declare(queue='test')
  15. client.wait(promise)
  16.  
  17. promise = client.basic_publish(exchange='', routing_key='test',
  18. body="Hello world!")
  19. client.wait(promise)
  20.  
  21. print " [*] Message sent"
  22.  
  23. promise = client.queue_declare(queue='test', passive=True)
  24. print " [*] Queue size:", client.wait(promise)['message_count']
  25.  
  26. promise = client.close()
  27. client.wait(promise)