Author Archives: Shang Erxin

About Shang Erxin

My undergraduate degree is Electrical Engineering and Automation. Graduate degree is Power Electronics and Power Drives. Personal favorite programming languages are C, C++, C# and Python. Currently I works as an architect/ software engineer/ programmer in Micron. During the free time, I also work as a free-lance programmer. Have good knowledge of, WCF, WPF, WF, Silverlight, Django, wxPython, jquery, SQL Server, SQLite, Mongo DB, Mobile development with Mono and Unity. If you want to find a free-lance programmer to solve problems with these techniques feel free to contact me.

Memory increasing issue when convert Pillow Image (8.3.1) either to Numpy array or OpenCV Image

Background introduction

Recently our project encounter a memory increase issue after upgrade from Pillow 8.2.0 to Pillow 8.3.0.

Memory consumption after upgrading the Pillow (8.3.1):

issue

The memory consumption before upgrading the Pillow (8.2.0):

normal

It seems the library bring lots of memory increments. BTW. The issue can not be produce before we build the project.

Solution

After I spent a few days to find the root cause and the solution. The issue code is located to a pillow image conversion

At the beginning, I suspect this would be a regression in Pillow, so I search their github issues

https://github.com/python-pillow/Pillow/issues/2019

https://github.com/python-pillow/Pillow/issues/5180

Both of these two threads indicate the issues are fixed. But I still tried to pull the fix code into our project. Without luck.

To keep the long story short, I grab two heap memory dump and compare with objects which increased most. This great help me reduce the search scope.

stack-trace

The relative issue code:

  1. np.array(pillow_image, [dtype=...])
  2. np.asarray(pillow_image)

 

All the above conversion will lead to the numpy array keep exist till a certain point trigger python release the accumulated objects.

Then I check the release note of the Pillow, there is only one place directly relative to numpy array. First I try to withdraw the changes directly in 8.3.1, no help. Second try to directly down grade to 8.3.0. Still not works.

Now, what? Checked Pillow API, Numpy API, OpenCV API again and also check their source. A idea pop up and write a workaround code

  1. numpy_array = np.frombuffer(pillow_image.tobytes(), dtype=np.uint8)

To convert the image array to OpenCV

  1. # 3 or 4 base on your image mode is RGB or RGBA
  2. numpy_array.shape = (*reversed(image.size), 3)
  3.  
  4. # If want to convert the image to BGR then call the OpenCV method
  5. # COLOR_RGB2BGR will required the image mode is 3
  6. numpy_array= cv2.cvtColor(numpy_array, cv2.COLOR_RGB2BGR)

It works! Now the memory consumption return to normal level.

fixed

There are several places directly use the image conversion, we should consider unify the similar logic into a utility class and add a wrapper for the third party library the isolation the influences.

Summary

Third party library upgrade is a risk to a product. We have to upgrade the library due to a security reason in this time. The Pillow changes are checked too, have found several suspicious code will cause the issue. Need time to test, maybe update the result later.

 

 

 

Tips for resolve pip install error for some of the python 2.7 package on window

Pip is a wide used python package management tool, it works properly most of the time. But occasionally there will be a few error when install some special package which may required local build after download.
Here are two common ways to resolve these kinds of error

1. Use a prebuild binary, the most common library could be download from here. But the links seems dead for a while.

2. Install the necessary Cpp compiler for python 2.7 on window from here

After install the compiler rerun the pip install again

Good luck!

Handling None Standard File Upload with TruClient

First let’s explain what is the “none standard file upload” for a website. Let a user upload a file is a very common use case for a website. The most common way to implement this would be

  1. <input type="file"/>

 

Then post the file content with form to the server. This is the standard way to upload a file with HTML. The standard way have several limitation. The most significant is difficult to customize the style for the upload file control.

So the none standard file upload come up with this requirement. The main idea is let the web designer customize the looking as they preferred. After the user click the control then dynamically create an input element or trigger a hidden input click in the background. This make the upload control looks much better, but it lead the feature hard for automation UI test. The common automation tools such as selenium(webdriver) and jmeter do not provide a elegant way to handle none standard upload. The reason is due to the input element is dynamically created which nearly impossible to locate it with CSS selector or XPath.

In TruClient 12.55+ which provide a easy way to handle none standard file upload.

Here is a simply code snippet for demo none standard file upload with jsfiddle at http://jsfiddle.net/erxin/o4menymo/

None standard file upload code sample

None standard file upload code sample

1. Drag a “General Object Action” step
2. Select the aim object which the user will click to upload the file
3. Change the step type from “Click” to “Upload”
4. Assign the right file path to the “Path” parameter
5. Done.
none-standard-upload

Currently the none standard file upload step only support in TruClient Chrome and IE. The Firefox version will coming soon.

Enjoy!