World Academy of Science, Engineering and Technology 1 2005
Automatic Camera Calibration for Images of Soccer Match

Qihe Li, Yupin Luo

AbstractCamera calibration plays an important role in the
domain of the analysis of sports video. Considering soccer video, in most cases, the cross-points can be used for calibration at the center of the soccer field are not sufficient, so this paper introduces a new automatic camera calibration algorithm focus on solving this problem by using the properties of images of the center circle, halfway line and a touch line. After the theoretical analysis, a practicable automatic algorithm is proposed. Very little information used though, results of experiments with both synthetic data and real data show that the algorithm is applicable.
KeywordsAbsolute conic, camera calibration, circular points,

line at infinity.

I. INTRODUCTION
ECENTLY, much interest has been caught in athlete tracking and automatic semantic annotation of sports videos. In both domains, camera calibration is a necessary stage in order to extract metric information from 2D images. For decades, there has been much work on camera calibration. Originally, it was studied in the photogrammetry community, while now, became a hotspot in computer vision. Generally, the techniques on calibration can be classified roughly into two categories[1]: Photogrammetric calibration, which are classic and require expensive apparatus and complex steps to acquire a precisely result. Self-calibration, which can find the intrinsic parameters without any calibration object but the geometric correspondence among a sequence of images. There still exist some other techniques such as vanishing points and pure rotation of camera. Usually, in soccer video analysis, images taken near the penalty area are easy to be calibrated since there are enough markers in the image such as the goal line, the flagpost and the cross points of the lines. By extracting these markers, sufficient information can be acquired for calibration, Szenberg et al. solved this problem in [2]. But for the center of the field (Fig. 2(a)), we can only get a halfway line, two touch lines (usually only the touch line near the camera is clear enough to be extracted) and the center circle in most cases. There are not more than four cross points on the same straight line can be
used, resulting in undermined equations; not only intrinsic and extrinsic parameters but also the homography matrix cannot be computed. Some work has been done on how to calibrate by using the image of the circle and straight lines: Meng et al.used a planar calibration pattern which includes a circle and a pencil of lines passing through the circles center[3]; Wu et al. used the images of two parallel circles[4], and Zhang et al. used two circles that intersect each other[5]. All of them used the properties of the absolute conic (or the absolute quadric) and the line at infinity. Some other methods used the property of straight lines[6], but usually, they can only compute the lens distortion parameters. All the methods mentioned above do not match our destination. In this paper we propose a new method let both intrinsic and extrinsic parameters can be computed if the noise and the distortion of the lens can be neglected. When the interference is so severe that the result cannot be convergence, at least the homography matrix can be computed and then an approximate solution can be acquired under some reasonable assumptions. The paper is organized as follows. Section II describes the basic algorithm of the method. The algorithm used in practice is offered in Section III. Section IV provides both synthetic and real data experimental results. Finally, conclusion is given. II. BASIC ALGORITHM A. Notation and Camera Model The notations used in this paper are presented below. We use a common letter to denote a scalar, a bold lowercase letter for a vector and a bold uppercase letter for a matrix, for example, x is a scalar; x is a vector and X is a matrix. In addition, XT denotes the transpose of X, X-1 for the inversion, and X-T for (X-1 )T or (XT )-1. A 2D point is denoted by m= [u v]T. A 3D point is denoted by n=[x y z]T. We use italic to denote their homogeneous coordinates, e.g. m = [u v t]T and n = [x y z t]T. The camera model used is the pinhole model that the imaging process from a 3D point n to its image m can be expressed[7] as Equation: km = K[R t]n (1), where K is the camera matrix including all the intrinsic parameters and denoted in Eq.(2). fu s u0 (2) 0 fv v0 K 1
Considering the 3D projective space, all the points satisfying the equation t = 0 are called points at infinity. They form the
Manuscript received November 4, 2004. Qihe Li is with the Department of Automation, Tsinghua University, Bejing 100084, P.R.China. Tel: +86-010-62792482, e-mail: lqh02@mails.tsinghua.edu.cn. Yupin Luo is with the Department of Automation, Tsinghua University, Beijing 100084, P.R.China. e-mail: luo@tsinghua.edu.cn.

plane at infinity. The absolute conic is the set of points in the plane at infinity which satisfies nTn = 0. By using Eq(1), we get the image of the absolute conic (IAC for short), which equals K-TK-1. Thus, we can derive all the intrinsic parameters by using Cholesky factorization with determined IAC. Since all the markers on the soccer field lie on the same plane, we can assume that Z coordinate of the model plane is 0 without loss of generality. By using ri to denote the ith column of R, from Eq(1) we have Eq(3)[1]: x u x x x k v K r1 r2 r3 t y K r1 r2 t y KM y H y (3) 0 t t t 1 t Based on the theory of projective geometry[7], all points at infinity on the model plane form the line at infinity, denoted by l. The cross points of the circle on the model plane and l are called circular points and denoted by I(1,i,0,0), J(1,-i,0,0). I and J lie on the absolute conic , so they satisfy the equation nTn = 0, thus the image of them must lie on IAC. Using Im and Jm to denote the image of I and J respectively, they should satisfy Eq(4): T T I m K -T K -1 I m 0 Re( I m K -T K -1 I m ) 0 (4) T T -T -1 J m K K J m 0 Im( I m K -T K -1 I m ) 0 Every image will generate 2 equations, and then we could get 6 equations through three images, therefore the five intrinsic parameters can be computed via least square method. Details of the pinhole model can be found in [7]. B. Computing the Center Point of the Center Circle

D C O B A c o b a d

The method used to solve equation of line is similar to that of ellipse. A line can be denoted as L(x, y) = Ax + Bx + C, and two points are enough to derive its equation[7]. Also, we can use more points considering the precision, thus, a least square solution can be acquired. D. Computing Circular Points After getting the the image of the circles center, the equations of the lines and the ellipse, we can calculate the circular points. There are two methods adoptable, that are described as follows respectively: Method I. This method is based on the method proposed by Wang [8]. It needs two rectangle or one square to be recognized, so as to acquire four points on the line at infinity, and then the circular points can be computed out. Since we have already got a diameter of the center circle, i.e., the halfway line, one more diameter perpendicular to the halfway line will enable us to acquire a square. The two touch lines are parallel to each other, and the halfway line is perpendicular to them. Thus, a line passing through the center of the circle and crossing the two touch lines at infinity or parallel to them must be the diameter we need. If only one touch line can be extracted, we can use the two tangents of the ellipse which pass though the points b and c as an alternative. Eqution l = Cx (7)can be used to compute the tangents where C is the coefficient matrix of ellipse and x is point of tangency[7]. Method II. This method is based on the method proposed by Meng [3]. It needs a circle and more than one diameters of the circle to be recognized. Since the two cross points of the circle and its diameter harmonically conjugate with respect to the center point of the circle and the point at infinity, the coordinate of the point at infinity can be computed(use Eq(8)). Two or more points at infinity can determine the line at infinity, thus we can calculate the circular points by finding the cross points between the circle and the line at infinity. ao ap ( ab, op ) : 1 (p is point at infinity) (8) bo bp All the straight lines passing through the center of a circle are diameters. Since we have got the center of the circle in advance, it is easy to get as many diameters as we want. Thus, the method can be used. Method III. Since more than four points which do not lie on one line has been acquired as described above, Zhangs method also can be used[1]. E. Computing the Intrinsic Parameters As mentioned above, if we have three or more images of the soccer field from different orientations, we can compute out every images circular points, and then get all the intrinsic parameters by using Eq(4) and Cholesky factorization. F. Computing Homography Matrix and Extrinsic Parameters Computing extrinsic parameters means computing the matrix M in Eq(3). Usually we use DLT to obtain our destination[7]. First, we will compute the homography matrix H. Then M can

(a) Model plane Fig. 1

(b) Image plane
As shown in Fig. 1(b), the points a, b, o, c, and d are images of A, B, O, C, and D in Fig. 1(a) respectively. Using the invariance of the cross ratio, i.e., Eq(5), we can derive the coordinates of the image of the circles center, namely, o. AO AC ao ac (5) ( AB, OC ) : : (ab, oc) BO BC bo bc In practice, point D is a redundancy.
C. Presentation of Ellipse and Lines One ellipse can be denoted by equation E(x,y)=ax2+bxy+cy2 +dx+ey+f. Five points on the ellipse are enough to determine its equation[7], but usually we use more points to get a more precise result as shown in Eq(6). Solving the equation, we can get the coefficient. a x12 x1 y1 y12 x1 yb x2 x2 y2 y2 x2 yc (6) Ac 0 (n 5) d xn xn yn yn xn yn 1 e f
be calculated by equation M = K -1 H (9), so r1, r2 and t are acquired. By using methods such as edge detection, noise acquired. By using r3 = r1 r2 (10), at last we get the ratation reduction and so on, an image as Fig. 2(c) is acquired. matrix, and figure out all the extrinsic parameters. Every pair of corresponding points between the soccer field and its image can provide two equations in Eq(11), 0 x1 yx1v1 y1v1 v1 (a) (b) (c) x1 y0 x1u1 y1u1 u1 h 0 Fig. 2 (a) Blue component (b) Grayscale image (c)Binary (11) xn yn xn un yn u n u n Extracting the ellipse and lines accurately from image is an T important step for our method. Here we can use the RANSAC (h h1 h2 h3 h4 h5 h6 h7 h8 h9 ) method [9] to obtain our destination. thus, at least four pairs of points not lying on one line are needed. Least sqrare method will be used if there are more B. Computing Parameters points. Having extracted the ellipse and lines, their cross points can In the orignal image , we can get no more than four points be easily acquired. Then we can compute the parameters by lying on one line in most cases. But after we compute the using the method described in Section II. intrinsic parameters, more points can be acquired. In fact, we find that under many circumstances, the IAC is Using the Method I and III in subsection D, we can get the not positive definite result in Cholesky decomposition cannot images of two cross points between the center circle and the be used since the noise and distortion exist. Finding the closest diameter perpendicular to the halfway line, enough points are positive definite solution is not appropriate as pointed out by acquired. This method uses the points computed in previous Hartley[7]. Thus, only the homography matrix can be steps, fast and simple. If we want more points or Method II is computed by using the method described in subsection F, adopted, the following method can be used. section II. If we need to compute the intrinsic paramethers, an Since the coordinates of the cross points between the center appoximate method can be used. Usually, following truth can circle and the halfway line(a diameter) is known in advance, be reasonble: the skew factor s is very small, the principal point should the angle between the halfway line and another diameter is not very far from the center of the image, and the rectangular be computed out, we can calculate the coordinates of the cross pixel is nearly a square. Thus we can assume that s = 0, fu/fv = 1 points between the diameter and the center circle. Since the and the principle point is the image center. The method circular points are already calculated out, their dual conic can proposed in[10] can be used to compute the focal length f using be acquired by using equation C* = IJ T + JI T (12), and then Eq.(14) the angle between two lines l and m can be computed by h1h2 h4 h5 f (14) following equation(13). h7 h8 l T C* m Where hi is shown in Eq.(11). Then we can get the (13) cos (l T C* l )(m T C* m ) approximate solution after an iterative optimization method By using Eq(9)(10)(11)(12)(13), all the extrinsic parameters which is described in details in[10].

can be figured out. III. AUTOMATIC CALIBRATION IN SOCCER VIDEO In using the method introduced in section II, the key problem is to recognize the markers. After extracting these elements, we can compute their equations, the coordinate of the center point of the circle, etc., and then get the calibration parameters. Our method is divided into three steps: preprocessing, extraction, and calculation.
A. Preprocessing and Extraction The objective of preprocessing is to enhance the image for easy to extract the markers(ellipse and lines). Classic methods of image processing combine the color information of the soccer field can give us a satifying result. We know that the color of soccer field is green(grass) and the markers white. Of course its green color cannot be pure green, and we find that its green approaches more to red than to blue. Then we use the blue component of the image(Fig. 2(a)) instead of the grayscale image (Fig. 2(b)), thus sharper contrast can be
IV. EXPERIMENTS RESULTS In this section, the proposed algorithm has been tested on both synthetic and real images for evaluating its robustness and accuracy.
A. Computer Simulation In the simulation, the cameras property is: fu = 1250, fv = 1100, s = 1.0, u0 = 380, v0 = 280. The image resolution is 720 540. Three model planes are used, and the rotation angles and translate vectors are: r1 = [7.5o, 12o, 0]T, t1 = [-135,-95,154]T, r2 = [22.5o,20o,5o]T, t2 = [-135,-95,150]T, r3 = [5.625o,15o,-5o]T, t3 = [-135,-95,145]T. A Gaussian noise with 0 mean and standard deviation is added to the projected image points. The noise level varies from 0.2 pixels to 4.0 pixels. Every results is the average of 100 independent trials. Note that, the relative errors of (u0, v0) are measured with respect to fu, as proposed by Triggs in[11]. And since s represent the tangent of the skew angle which commonly is so small that the value of s is very close to the angle itself, its relative error is also
measured with respect to fu. We find that the effects of the three methods are very similar, so Fig. 3 shows only the result of Method II. TABLE 1 shows the absolute value of some of the results. Since the algorithm is too dependent on the precision of the coordinate of the image of the circles center point, the robustness is a little worse than Mengs [3].
errors and lens distortion make IAC not positive definite resulting in parameters cannot be calculated, at least the homography matrix can be computed, thus, an approximate solution can be acquired. If intrinsic parameters are known in advance, only the images of a circle and one of its diameters are needed for extrinsic parameters calibration. The shortcoming is the distortion parameters cannot be calibrated. In addition, the speed cannot satisfy the real-time requirements since RANSAC method is adopted. In the future, we will try to use the deformation of straight lines on the image to eliminate the distortion before calibration. Besides, the robustness will be considered further. Increasing the speed is also a direction of work.

Fig. 3 Relative errors of Method II TABLE 1 CALIBRATION RESULTS AT DIFFERENT NOISE OF METHOD II Noise level 0.2 0.6 1.0 1.6 2.0 3.0 fu fv s u0 v0 Fig. 4 (a) Images of a model plane contains simulation markers (b) (c) other two images of soccer field TABLE 2 RESULTS OF REAL DATA Paramete rs fu fv s u0 v0 Model plane Zhangs 1924.34 1724.95 3.36 376.46 266.39 Ours 2092.21 1784.360 273.60 e(%) 8.72 3.44 0.18 0.86 0.37 Zhangs 617.27 539.48 5.37 351.27 266.01 Soccer Field Ours 702.75 591.84 5.14 367.67 214.98 e(%) 13.78 9.71 0.04 2.66 8.27
1250.6 1249.5 1253.9 1262.6 1261.5 1280.3
1100.4 1099.1 1103.5 1112.1 1110.2 1126.9
1.1691 1.071 2.5642 5.9256 7.0858 16.179
380.25 377.64 381.19 385.37 379.26 381.88
279.48 280.52 276.11 268.85 270.86 254.21
B. Real Data The images used were taken by a Panasonic NV-MX2500 video camera whose resolution is 720 480. We use images of a model plane contains a pattern like the markers on the soccer field(only one image shown in Fig. 4(a)) first, and then images of a real soccer field(Fig. 4(b)(c)) are used. For comparison, Zhangs method[1] is used to calibrate the same camera. And we find that only the approximate result can be acquired(TABLE 2, the column e(%) means the releative error with respect to the result of Zhangs method). The relative errors are computed as described in subsection A. The reason of why the robustness in the computer simulation experiment is good but deteriorated when real images are used is that the adoption of least square method to fitting the equations of ellipse and lines can minimize the infection of Gaussian noise but cannot be effective when measurement errors and lens distortion exist. We can see that the relative error of the images of soccer field is greater than the model planes because the using of a wide-angle lens (Panasonic VW-LW 4307M) which results in much greater distortion than that of the images of model plane.

REFERENCES

[1] Z. Zhang, "A flexible new technique for camera calibration," Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 22, pp. 1330-1334, 2000. [2] F. Szenberg, P. C. P. Carvalho, and M. Gattass, "Automatic Camera Calibration for Image Sequences of a Football Match," presented at ICAPR 2001, pp. 301, Rio de Janeiro, Brazil, 2001. [3] X. Meng and Z. Hu, "A new easy camera calibration technique based on circular points," Pattern Recognition, vol. 36, pp. 1155-1164, 2003. [4] Y. Wu, H. Zhu, Z. Hu, and F. Wu, "Camera Calibration from the Quasi-affine Invariance of Two Parallel Circles," ECCV 2004, Prague, Czech Republic, pp. 190-202, 2004. [5] B.-W. Zhang, D. Liang, and F.-C. Wu, "Camera self-calibration based on circle," presented at Visualization and Optimization Techniques, pp. 286-289, Wuhan, China, 2001. [6] B. Prescott and G. F. McLean, "Line-Based Correction of Radial Lens Distortion," Graphical Models and Image Processing, vol. 59, pp. 39-47, 1997. [7] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision: Cambridge University Press, 2000. [8] G. Wang, F. Wu, and Z. Hu, "Novel approach to circular-points-based camera calibration," presented at Second International Conference on Image and Graphics, pp. 830-837, Hefei, China, 2002. [9] M. A. Fischler and R. C. Bolles, "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography," Graphics and Image Processing, vol. 24, pp. 381-395, 1981. [10] C. Chatterjee and V. P. Roychowdhury, "Algorithms for coplanar camera calibration," Machine Vision and Applications, vol. 12, pp. 84-97, 2000. [11] B. Triggs, "Autocalibration from Planar Scenes," presented at ECCV'98, pp. 89, Freiburg, Germany, 1998.

V. CONCLUSION AND FUTURE WORK The algorithm presented here is applicable for practical use. Though only a few markers can be acquired which means the camera cannot be calibrated by traditional method, we can compute the calibration parameters by using the properties of the images of ellipse and line. The robustness is good when only Gaussian noise exists, even if other noises, measurement