Experiments
My experiments are based on two phases. In the first phase I use synthetic image, which is taken from the pattern and modified by a variety of filters. The second phase is a live demonstration of the pen in action. I used projection transformation for all my experiments.
Phase 1: Synthetic
The first tests use single line, which resorts to using angle and scale provided from Hough Transform.
Figure 11: Single line with translation, scaling and rotation
In Figure 11 I can see the input images on the left side of each displayed image and the stitched version on the right side. The T is visible and equally sampled without difference between all the angles. This tells me that my linear solution performs well under translation, scaling and rotation.
Figure 12: Two lines with translation, scaling, rotation and shearing
In Figure 12 I have added shearing, which requires two lines and non-linear transformation to properly solve and stitch the image. In these cases I provide all the necessary information in the input image for a good solution. On the right side of each displayed image you can see that image being translated, scaled, rotated and sheared into the right place. The lines appear parallel and continuous as they are supposed to.
Figure 13: Two lines with Gaussian noise at 15%, 30% and 50% respectively
In Figure 13 I have tried different amounts of Gaussian noise until the algorithm failed. My experiments showed that even with a lot of noise present (30-40%) the algorithm was still able to properly stitch the image. However, the visual appearance of such image in stitched form is not so pleasant. The algorithm has failed at 50% noise, which is most likely due to the fact that the difference between the gray and the black pixels that represent the pattern have blended together too much.
Phase 2: Real
The
real camera testing consisted of two main parts. In the first part I
allow both linear and non-linear transforms. This produces a lot of
visual error since warping and shearing are left unchanged. See Figure 14, Figure 15
and Figure
16 for experimental
results.
Figure
14: Linear and non-linear with no text
Figure
15: Linear and non-linear with CS498
text
The Figure 15
demonstrates why linear transform produces unsatisfactory results, because
it is able to align only one line and ignores the second. Particularly
we can see that “CS” part was aligned using the bottom line while
“498” part was aligned using the top line. This produced an undesired
effect where it appears the text is not aligned. The speed was the good
part where I was able to average 4-8 lines per second. The next figure
illustrates that this approach is able to capture all the parts of the
image even where lines pass through the pattern. This is due to the
fact that the algorithm is fairly flexible and just needs to find two
adjacent numbers anywhere in the image in order to stitch it.
Figure
16: Non-linear and linear with text and pictures.
In the second part I have tested using only non-linear transform for stitching. This part also increased the reliability of detected lines since the transform requires a minimum of 4 numbers. The down side is that one image is detected every 3-5 seconds due to restricted positioning of the pen that must fit two lines into one image. See Figure 17 for scan of the same text.
Figure
17: Non-linear with CS498 text
In this figure we can see that the results are very good and all the reference lines appear properly aligned to their relative lines in the original pattern. It took multiple passes over the area to capture the image, which is explained by the increase amount of information necessary for non-linear transform. The algorithm fails to find parts of the image where lines pass over the pattern, because there is simply not enough information in each image to find the 4 points for non-linear transform.
Files
The following is a zip file of the synthetic inputs I have used for this project. They are pretty much unit tests for each part of the algorithm since I have used them in order of evolution of the project. What I mean is that I only scaled the pattern when I wanted to detect edges, lines and pattern. Then I rotated it when I wanted to test the Resize and Rotate algorithms. Finally, I sheared the pattern to test projection and bilinear transforms.