1: \section{Performance Analysis}
2: \label{sec-analysis}
3: To analyze the performance, we introduce the following definitions in
4: for 1-hop neighborhood establishment (these definitions follow the
5: classical error terminology of hypothesis testing in statistics).
6:
7: \begin{definition}
8: If two nodes $p$ and $q$ are not 1-hop neighbors, and
9: they are considered by the algorithm to be 1-hop neighbors,
10: then the algorithm is said to make a \emph{type A error} for
11: $(p,q)$.
12: \end{definition}
13:
14: \begin{definition}
15: If two nodes $p$ and $q$ are 1-hop neighbors, and
16: they are not classified by the algorithm as 1-hop neighbors,
17: then the algorithm is said to make a \emph{type B error} for
18: $(p,q)$.
19: \end{definition}
20:
21: RSSI ranging and the fuzzy membership functions effectively have
22: thresholds between classifying a pair $(p,q)$ as 1-hop neighbors or
23: not. Moving this threshold one way or another can change the number
24: of type A and type B errors.
25:
26:
27: For our field experiment, among 121 1-hop pairs in the grid,
28: we had 17 type B errors and we had 9 type A errors in 192 2-hop pairs.
29: Among all the 50 nodes, 24 nodes have correct 1-hop neighborhoods.
30: In other words, with the parameters we chose for the
31: fuzzy membership functions, we recognized 86.0\% of the 1-hop neighbors
32: and took 4.7\% of 2-hop neighbors for 1-hop neighbors.
33:
34: Our experiment shows that we manage
35: to reduce the number of type A and type B errors using
36: fuzzy membership functions
37: proposed in section~\ref{sec-algorithm}. This result is obtained though
38: RSSI ranging data is inaccurate. We observed that even
39: with 30 messages in each group,
40: the mean RSSI reading over the same distance could
41: have a quite large variance. Distance estimation using RSSI alone
42: can generate error as large as 60\%.
43: In our experiment, mean RSSI reading of one group
44: of messages over distance of 10 meters can be
45: same as the overall average reading over the distance of 16 meters.
46: In fact, when we tried to
47: establish the 1-hop neighborhood using RSSI distance estimation alone,
48: and keep the type A error ratio at the similar level,
49: we ended up having many more type B errors.
50: The type A error ratio for 2-hop neighbors is 5.4\%, and ratio of
51: correctly recognized 1-hop neighbors
52: is 72.2\%.
53:
54: Both type A and type B errors in the 1-hop neighborhood establishment can
55: affect the tuples of the nodes. But their effects are not the same.
56: In our topology, for most pairs of nodes, there are at least two
57: paths of minimum length.
58: If a type B error occurs and breaks one of the shortest paths, the distance
59: calculation is more likely to give the same result because there are other
60: shortest paths. In the mean time, a type A error is more likely to change
61: the length of the shortest path. So we can see a type A error is more likely
62: to affect the 4-tuple of a node. Thus we have the following:
63: \begin{quote}
64: {\bf Observation.}
65: The table lookup algorithm tolerates more type B
66: errors than type A errors.
67: \end{quote}
68: For the algorithm to achieve satisfying results,
69: we need as few errors as possible.
70: Unfortunately, there is a trade-off between the number
71: of type A and type B errors.
72: Depending on how to choose a fuzzy membership function, the
73: score assignment and the rules to pick 1-hop number, we can
74: reduce the number of one type of errors, but the
75: number of the other type will increase.
76:
77: We also ran simulations by injecting type A and type B errors.
78: It turns out that to obtain good localization result,
79: we need to reduce the number of type A errors to be fewer
80: than 12, and number of type B errors to be fewer than 31
81: in the 50 node grid.
82:
83: We explored the effect of segment size on the algorithm by simulations.
84: We conducted simulations for larger segments, assuming that we could get 1-hop
85: neighborhood information of same accuracy as
86: in our experiment. We did simulations on segments with different sizes
87: ranging from 50 to 200(50, 60, 70, 80, 90, 100, 150, 200). All
88: simulations use only four anchors. For each segment
89: size, we ran 1000 simulations.
90: Figure~\ref{ratecChart} and \ref{avgErrorChart} shows $rate_c$ and
91: $avg_{error}$ for different segment sizes.
92:
93:
94: \begin{figure}[ht]
95: \centering
96: \includegraphics {ratec.eps}
97: \caption{$rate_{c}$ for different segment sizes}
98: \label{ratecChart}
99: \end{figure}
100:
101: \begin{figure}[ht]
102: \centering
103: \includegraphics {avgError.eps}
104: \caption{$avg_{error}$ for different segment sizes}
105: \label{avgErrorChart}
106: \end{figure}
107:
108:
109: With a larger segment, the accuracy of the algorithm decreases. This is
110: because with longer paths, the errors in grid distance counting
111: will accumulate.
112: