1: \section{Experimental Evaluation}\label{eval}

2: \label{experiments}

3: \rednote{Can you write an outline of what experiments/data you are

4: planning to represent?  Experimental planning is a critical part of a

5: good paper.}

6: 

7: In this section, we present an experimental evaluation exploring the

8: performance of TARMAC under different conditions, and compare it

9: against two existing schemes for protection against traffic analysis

10: in sensor networks\cite{deng-intrusion,deng-antitrafficanalysis}; we

11: call these solutions Intrusion 1 and Intrustion 2 respectively.  We

12: also compare against an unprotected network to evaluate the overhead

13: necessary for camouflaging traffic.  We implemented TARMAC in network

14: simulator NS2 (version 2.29) \cite{ns2}, by extending IEEE 802.11 MAC

15: protocol according to our design.  To enable fair comparison, we also

16: implemented Intrusion 1 and 2 on NS2 (they were implemented on TinyOS

17: originally).

18: 

19: 

20: 

21: 

22: 

23: 

24: 

25: 

26: 

27: 

28: 

29: 

30: 

31: 

32: 

33: 

34: 

35: 

36: 

37: 

38: 

39: 

40: 

41: 

42: 

43: 

44: In our experiments, 100 sensors are regularly deployed in $10\times10$ grids covering an

45: area of $200\times200 meter^2$, in which each node is located at the center of each grid.

46: The table \ref{tab:simpara} shows some simulation parameters general for all studies.

47: Other simulation parameters are summarized in Table~\ref{tab:simpara} unless

48: explicitly stated.

49: 

50: \begin{table}

51: \centering

52: \begin{tabular}{|c|c|}

53: \hline Parameter & value \\

54: \hline No. of Data Sources & 100 \\

55: \hline Traffic type & CBR\\

56: \hline CBR packet size & 32B\\

57: \hline Transmission Range & 40 meters\\

58: \hline BandWidth & 2Mbps\\

59: \hline Routing Period & 5 seconds\\

60: \hline Traffic Period & 100 seconds \\

61: \hline Simulation Period & 400 seconds \\

62: \hline TARMAC slot size & 64 Byte\\

63: \hline

64: \end{tabular}

65: \caption{Some simulation parameters}

66: \label{tab:simpara}

67: \end{table}

68: 

69: 

70: \subsection{Analysis of Basic TARMAC}

71: 

72: \begin{figure}[h]

73: \centering

74: \includegraphics[width=0.45\textwidth]{busslots-dr}

75: \caption{TARMAC Delivery Ratio Study}

76: \label{fig:busslots-dr}

77: \end{figure}

78: \begin{figure}[h]

79: \centering

80: \includegraphics[width=0.45\textwidth]{busslots-occupacy}

81: \caption{TARMAC Bus Occupation Study}

82: \label{fig:busslots-occupacy}

83: \end{figure}

84:  \begin{figure}[h]

85:  \centering

86:  \includegraphics[width=0.45\textwidth]{busslots-coll}

87:  \caption{TARMAC Collision Ratio Study}

88:  \label{fig:busslots-coll}

89:  \end{figure}

90: 

91: Figure \ref{fig:busslots-dr} shows the delivery ratio of shortest path

92: routing on TARMAC as a function of the size of the frame and the

93: transmission period. When TARMAC has a small period, even when the

94: buses are small, the delivery ratio is high since the collision ratio

95: is low as bigger frames suffer more collisions

96: (Figure~\ref{fig:busslots-coll}).  When the period is large, if the bus

97: size is small, then the effective capacity is not sufficient to carry

98: the packets leading to increased packet delay and reduced delivery

99: ratio.

100: 

101: When the bus size increases, more collision happen and more packets

102: are dropped due to collisions. For those with longer periods, the

103: delivery ratio increases since each frame carries more packets.  After

104: some point, the collision ratio dominates as the physical capacity of

105: the channel is reached. Figure \ref{fig:busslots-occupacy} shows the

106: occupation ratio of TARMAC bus packets (effective packets / number of

107: slots). As can be expected, the longer the transmission period, the

108: higher the bus occupation ratio.

109: \begin{figure}[h]

110: \centering

111: \includegraphics[width=0.45\textwidth]{busslots-delay}

112: \caption{TARMAC Average Delay Study}

113: \label{fig:busslots-delay}

114: \end{figure}

115: Figure \ref{fig:busslots-delay} shows the average delay of all

116: effective packets.  As the transmission period increases, the time to

117: get from source to destination increases. The smaller the buses are,

118: the longer is the time needed to reach the destination as packets wait

119: at each intermediate node.  Figure \ref{fig:busslots-enpkt} shows the

120: energy consumption per packet.  More energy is expended when the

121: period is reduced (sending more often) or when the frame size is

122: increased.

123: 

124: 

125: 

126: 

127: \begin{figure}[h]

128: \centering \includegraphics[width=0.45\textwidth]{busslots-enpkt}

129: \caption{TARMAC Energy Consumption Study}

130: \label{fig:busslots-enpkt}

131: \end{figure}

132: 

133: 

134: 

135: 

136: 

137: 

138: 

139: 

140: 

141: 

142: \subsection{TARMAC with different traffic patterns}

143: 

144: In this section, we study the performance of TARMAC under different

145: traffic patterns.  In the same grid scenario, we change the number of

146: data sources and their distribution in the networks, considering the

147: following 3 cases: all nodes as data sources, $\dfrac{1}{3}$($\sim

148: 35$) nodes as data sources and a quarter of nodes in the same corner

149: are data sources. We also study two CBR send rates: 2 packets per

150: second and 1 packet per second.

151: 

152: \begin{figure}[h]

153: \centering

154: \includegraphics[width=0.45\textwidth]{busptn-dr}

155: \caption{TARMAC Delivery Ratio Study}

156: \label{fig:busptn-dr}

157: \end{figure}

158: \begin{figure}[h]

159: \centering

160: \includegraphics[width=0.45\textwidth]{busptn-delay}

161: \caption{TARMAC Delay Study}

162: \label{fig:busptn-delay}

163: \end{figure}

164: \begin{figure}[h]

165: \centering

166: \includegraphics[width=0.45\textwidth]{busptn-enpkt}

167: \caption{TARMAC Average Energy Consumption Study}

168: \label{fig:busptn-enpkt}

169: \end{figure}

170: 

171: 

172: 

173: 

174: 

175: 

176: 

177: 

178: Figure \ref{fig:busptn-dr} shows the delivery ratios of TARMAC with

179: different traffic patterns.  As can be expected, the higher the data

180: rate, the lower the delivery ratio.  The capacity of the bus also

181: impacts the delivery ratio.  The delivery ratio of pattern with 35

182: nodes is higher than that of only 25 nodes. The reason is that those

183: 25 nodes are in the same crowded area creating hotspotting along the

184: path to the basestation.  The average delay and energy consumption are

185: shown in figures \ref{fig:busptn-delay}, \ref{fig:busptn-enpkt} and

186: follow expectations.

187: 

188: 

189: \subsection{TARMAC vs Intrusion 1}

190: 

191: \begin{figure}[h]

192: \centering

193: \includegraphics[width=0.45\textwidth]{bus-intrusion1_dr}

194: \caption{Delivery Ratio}

195: \label{fig:bus-intrusion1_dr}

196: \end{figure}

197: \begin{figure}[h]

198: \centering

199: \includegraphics[width=0.45\textwidth]{bus-intrusion1_en}

200: \caption{Energy Consumption Per effective Packet}

201: \label{fig:bus-intrusion1_en}

202: \end{figure}

203: \begin{figure}[h]

204: \centering

205: \includegraphics[width=0.45\textwidth]{bus-intrusion1_trans}

206: \caption{Transmission Cost ($M/m$)}

207: \label{fig:bus-intrusion1_trans}

208: \end{figure}

209: \begin{figure}[h]

210: \centering

211: \includegraphics[width=0.45\textwidth]{bus-intrusion1_transize}

212: \caption{Transmission Cost ($S/m$)}

213: \label{fig:bus-intrusion1_transize}

214: \end{figure}

215: \begin{figure}[h]

216: \centering

217: \includegraphics[width=0.45\textwidth]{bus-intrusion1_delay}

218: \caption{Average End-to-End Transmission Delay}

219: \label{fig:bus-intrusion1_delay}

220: \end{figure}

221: 

222: In this study, we compare the performance of TARMAC and the Intrusion

223: 1 solution~\cite{deng-intrusion}.   We select one node as a data sink in

224: the up-left corner of the simulation area.  Each node in the network

225: generates a CBR data traffic to the sink with a 2 packets per

226: second rate.

227: 

228: \rednote{Is the next commented stuff still true?}

229: 

230: 

231: 

232: 

233: 

234: 

235: Figure \ref{fig:bus-intrusion1_dr} shows the overall delay of TARMAC

236: and Intrusion 1, against that of Shortest Path routing which serves as

237: an upper bound on performance.  As we can see, the buffer size affects

238: the performance of intrusion 1 as well as bare SP. But TARMAC is not

239: affected apparently by the buffer size. The reason may be due to the

240: broadcast nature of TARMAC, and the unicast nature of intrusion 1 and

241: SP.  Since they use unicast, both intrusion 1

242: and SP may forward packet with best effort, holding packets for

243: possible retransmission until success. TARMAC only holds packet for

244: next available bus. So if the transmission period is low enough,

245: the required buffer size is not big.

246: 

247: Figure \ref{fig:bus-intrusion1_en} shows a somewhat surprising result

248: about the energy consumption per data packet received by the data

249: sink. The energy consumption per effective packet of TARMAC is nearly

250: the same as the ideal optimal solution. However, this result is

251: reasonable since each data packet is transmitted at each hop only

252: once, requiring nothing more than itself (in term of

253: transmission). Another reason leads to this result is the lower

254: delivery ratio. Some packets were dropped at the early stage of its

255: traveling from source to sink. We can reach this conclusion more

256: directly from Figure~\ref{fig:bus-intrusion1_trans} and

257: \ref{fig:bus-intrusion1_transize}.  Since unicast transmission used by

258: intrusion and SP requires control packets (RTS/CTS, ACK, etc.), the

259: number of packets transmitted physically is higher than that with

260: broadcasting: with TARMAC, all packets --routing, arp, data-- are

261: transmitted through broadcasting. $M/m$ refers to the total number of

262: packets transmitted physically (MAC layer packets) over the number of

263: effective packets (data packets received by sink).  $S/m$ indicates

264: the total size (in bytes) of packets transmitted physically over the

265: size of effective packets.

266: 

267: Figure \ref{fig:bus-intrusion1_delay} shows the average end-to-end

268: delay of all effective packets. The average delay of TARMAC is shorter

269: than it of SP may due to the lower delivery ratio (most packets with

270: longer delay are dropped before reaching the sink).

271: 

272: \subsection{TARMAC vs Intrusion 2}

273: \begin{figure}[h]

274: \centering

275: \includegraphics[width=0.45\textwidth]{bus-intrusion2_dr}

276: \caption{Delivery Ratio}

277: \label{fig:bus-intrusion2_dr}

278: \end{figure}

279: \begin{figure}[h]

280: \centering

281: \includegraphics[width=0.45\textwidth]{bus-intrusion2_en}

282: \caption{Energy Consumption Per effective Packet}

283: \label{fig:bus-intrusion2_en}

284: \end{figure}

285: 

286: Intrusion 2~\cite{deng-antitrafficanalysis} does not buffer incoming

287: packets: only one packet is held for forwarding at each hop. Each node

288: keeps sending the held packet until it overhears the forwarding of the

289: corresponding packet at the next hop. Buffer size does not affect

290: the performance of TARMAC as well as was shown in the previous

291: section.%%  So we compare intrusion 2 with TARMAC with respect to the

292: 

293: 

294: 

295: 

296: Figure \ref{fig:bus-intrusion2_dr} shows the delivery ratio of TARMAC

297: and intrusion 2 with respect to the fixed transmission rate. The black

298: straight line stands the delivery ratio of ideal optimal solution

299: (SP). As the fixed transmission rate decreases, the delivery ratio

300: decreases as well since the capacity of the bus decreases.  As we can

301: see if the buffer size is double of TARMAC, the performance is

302: considerably better. Intrusion 2 can not benefit the increasing of the

303: buffer size due to its design is just holding one packet. Figure

304: \ref{fig:bus-intrusion2_en} shows the energy consumption per effective

305: packet.  The situation is the same as we discussed before. Intrusion 2

306: tries to retransmit a data packet many times at each hop through the

307: forwarding path. Meanwhile, no matter what the buffer size is, TARMAC

308: just forwards each packet once at each hop.

309: 

310: \begin{figure}[h]

311: \centering

312: \includegraphics[width=0.45\textwidth]{bus-intrusion2_trans}

313: \caption{Transmission Cost ($M/m$)}

314: \label{fig:bus-intrusion2_trans}

315: \end{figure}

316: \begin{figure}[h]

317: \centering

318: \includegraphics[width=0.45\textwidth]{bus-intrusion2_transize}

319: \caption{Transmission Cost ($S/m$)}

320: \label{fig:bus-intrusion2_transize}

321: \end{figure}

322: 

323: \begin{figure}[h]

324: \centering

325: \includegraphics[width=0.45\textwidth]{bus-intrusion2_delay}

326: \caption{Average End-to-End Transmission Delay}

327: \label{fig:bus-intrusion2_delay}

328: \end{figure}

329: 

330: Figure \ref{fig:bus-intrusion2_delay} shows the average end-to-end

331: delay of all effective packets. The straight line at bottom is that of

332: the ideal solution (SP).  At first, the average delay of TARMAC with

333: all buffer sizes are shorter than that of SP, perhaps due to the lower

334: delivery ratio: most data packets with longer delay were dropped. As

335: the buffer sizes increase, the average delay of TARMAC increases.

336: 

337: 

338: 

339: 

340: 

341: 

342: 

343: 

344: 

345: 

346: 

347: 

348: 

349: 

350: 

351: 

352: 

353: 

354: 

355: 

356: 

357: 

358: 

359: 

360: 

361: 

362: 

363: 

364: 

365: 

366: 

367: 

368: 

369: 

370: 

371: 

372: 

373: 

374: 

375: 

376: 

377: 

378: 

379: 

380: 

381: 

382: 

383: 

384: 

385: 

386: 

387: 

388: 

389: 

390: 

391: 

392: 

393: 

394: 

395: 

396: 

397: 

398: 

399: 

400: 

401: 

402: 

403: 

404: 

405: 

406: 

407: 

408: 

409: 

410: 

411: 

412: 

413: 

414: 

415: 

416: 

417: 

418: 

419: 

420: 

421: 

422: 

423: 

424: 

425: 

426: 

427: 

428: 

429: 

430: 

431: 

432: 

433: 

434: 

435: 

436: 

437: 

438: 

439: 

440: 

441: 

442: 

443: 

444: 

445: 

446: 

447: 

448: 

449: 

450: 

451: 

452: 

453: 

454: 

455: 

456: 

457: 

458: \subsection{Evaluation of Multipath Routing and Adaption of TARMAC frame frequency}

459: The routing layer is orthogonal to TARMAC, however, if it makes uses TARMAC it

460: can greatly increase its performance. Let us assume that the routing layer just

461: implements, say, shortest path routing. Now, if there is an event at a certain

462: location and all the nodes at that location start sending data suddenly, the

463: capacity of the network will be reached and the queues would fill up causing packets

464: to be dropped. In addition to this, the empty TARMAC frames from nodes adjacent to

465: the data flow would interfere with the data flow causing further drops. However,

466: if the routing layer took advantage of the presence of TARMAC, it could employ multi-path

467: routing. This would mean that the frames that were originally empty would now hold data packets

468: and portions of the data would follow different paths to get to the destination. This would

469: increase the delivery ratio, reduce the load on the network capacity and decrease the overhead

470: as now the frames are carrying packets rather than being empty.

471: \begin{figure}[h]

472: \centering

473: \includegraphics[width=0.45\textwidth]{mp-adapt-dr}

474: \caption{Multipath Routing and Adaption gain}

475: \label{fig:mp-adapt-dr}

476: \end{figure}

477: The figure \ref{fig:mp-adapt-dr} shows the effects of multipath routing and the

478: adaption of frame frequency. The adaption and multipath routing helps more packets passing

479: through the network if density of data sources are high. The does not help to much (or

480: even hurt) those with lower data sources density, since more collision may happen.

481: 

482: 

483: