# Difference between revisions of "Quantization - Real-World Evaluation"

(Created page with "For the evaluation of the quantization schemes we used bidirectional channel measurements of campaigns from related works <ref>[229]</ref>. The channel measurement protocol wa...") |
|||

Line 2: | Line 2: | ||

Unfortunately, detailed information on the generation of the RSSI values are not specified in the standard and also not provided by the vendors. The fact that each manufacturer can provide proprietary implementations of the RSSI generation makes real-world security analyses difficult. For example, the potential of RSSI manipulation of each realization needs to be evaluated. | Unfortunately, detailed information on the generation of the RSSI values are not specified in the standard and also not provided by the vendors. The fact that each manufacturer can provide proprietary implementations of the RSSI generation makes real-world security analyses difficult. For example, the potential of RSSI manipulation of each realization needs to be evaluated. | ||

The set of channel profiles originates from an indoor office environment and an outdoor ’inner yard’ environment. For each environment several positioning were produced. Details of the positionings are given in the paper. Each measurement consists of both legitimate parties as well as a passive attacker. Results of attackers correlated observation are given in the paper as well. | The set of channel profiles originates from an indoor office environment and an outdoor ’inner yard’ environment. For each environment several positioning were produced. Details of the positionings are given in the paper. Each measurement consists of both legitimate parties as well as a passive attacker. Results of attackers correlated observation are given in the paper as well. | ||

− | [[File:Qrwe-image1.png|450px|thumb|center|Figure 6.21: Simulation results for IKGR versus correlation coefficient.]] | + | [[File: Qrwe-image1.png|450px|thumb|center|Figure 6.21: Simulation results for IKGR versus correlation coefficient.]] |

− | [[File:Qrwe-image2.png|200px|thumb|right|Figure 6.22: Key extraction efficiency of different quantization schemes and parameters. The entropy of the raw channel profiles is given in red. The entropy of the preliminary key material is given in blue. The mutual information of both random variables is given in violet.]] | + | [[File: Qrwe-image2.png|200px|thumb|right|Figure 6.22: Key extraction efficiency of different quantization schemes and parameters. The entropy of the raw channel profiles is given in red. The entropy of the preliminary key material is given in blue. The mutual information of both random variables is given in violet.]] |

− | [[File:Qrwe-image3.png|270px|thumb|right|Table 6.1: Information-theoretic evaluation results of different quantizations schemes. C - channel profiles, Q - preliminary key material.]] | + | [[File: Qrwe-image3.png|270px|thumb|right|Table 6.1: Information-theoretic evaluation results of different quantizations schemes. C - channel profiles, Q - preliminary key material.]] |

Based on all data of the large measurement campaign (with approximately 112 hours of measurement and 40 000 000 RSSI values per channel) we evaluated the BER versus correlation coefficient. Therefore, we calculated the blockwise correlation as well as the corresponding blockwise BER and sorted those by correlation value. Further, we calculated the BER distribution for the following subgroups: [0 : 0.05, 0.05 : 0.1, ..., 0.95 : 1]. | Based on all data of the large measurement campaign (with approximately 112 hours of measurement and 40 000 000 RSSI values per channel) we evaluated the BER versus correlation coefficient. Therefore, we calculated the blockwise correlation as well as the corresponding blockwise BER and sorted those by correlation value. Further, we calculated the BER distribution for the following subgroups: [0 : 0.05, 0.05 : 0.1, ..., 0.95 : 1]. | ||

Line 19: | Line 19: | ||

The practice-oriented results of the schemes using Lloyd-max, mean threshold, and median threshold do not provide significant differences to the simulation results. | The practice-oriented results of the schemes using Lloyd-max, mean threshold, and median threshold do not provide significant differences to the simulation results. | ||

− | [[File:Qrwe-image4.png|270px|thumb|right|Figure 6.23: Information-theoretic performance results of the channel profiles and the initial key material. The evaluation results are based on real-world measurements (<math>\rho\approx</math> 0.97). | + | [[File: Qrwe-image4.png|270px|thumb|right|Figure 6.23: Information-theoretic performance results of the channel profiles and the initial key material. The evaluation results are based on real-world measurements (<math>\rho\approx</math> 0.97). |

The entropy of the channel profiles (red), the entropy of the quantizer’s output bits | The entropy of the channel profiles (red), the entropy of the quantizer’s output bits | ||

(blue), and the mutual information (overlapping) are illustrated.]] | (blue), and the mutual information (overlapping) are illustrated.]] | ||

− | [[File:Qrwe-image5.png|400px|thumb|center|Figure 6.24: Evaluation results based on the real-world setups 1−12 of all quantization schemes for (a) estimated min-entropy over time, (b) on-line entropy estimation for different | + | [[File: Qrwe-image5.png|400px|thumb|center|Figure 6.24: Evaluation results based on the real-world setups 1−12 of all quantization schemes for (a) estimated min-entropy over time, (b) on-line entropy estimation for different |

downsampling factors.]] | downsampling factors.]] | ||

==References== | ==References== | ||

<references/> | <references/> |

## Revision as of 14:24, 25 October 2017

For the evaluation of the quantization schemes we used bidirectional channel measurements of campaigns from related works ^{[1]}. The channel measurement protocol was implemented on a mobile and portable hardware platform. The credit-card-sized computer was equipped with a battery for mobility as well as with the TL-WN722N and the CC2531EMK wireless USB adapter, utilizing IEEE 802.15.4 (for applications like ZigBee ,WirelessHART, 6LoWPAN) and IEEE 802.11 (for Wi-Fi), both at 2.4 GHz. The protocol presented ensures synchronized channel measurements (RSSI values on a per-packet basis) between each of the three parties within the probing duration of 5 ms. The sampling rate is 100 measurements per second for subsequent rounds of channel probing.
Unfortunately, detailed information on the generation of the RSSI values are not specified in the standard and also not provided by the vendors. The fact that each manufacturer can provide proprietary implementations of the RSSI generation makes real-world security analyses difficult. For example, the potential of RSSI manipulation of each realization needs to be evaluated.
The set of channel profiles originates from an indoor office environment and an outdoor ’inner yard’ environment. For each environment several positioning were produced. Details of the positionings are given in the paper. Each measurement consists of both legitimate parties as well as a passive attacker. Results of attackers correlated observation are given in the paper as well.

Based on all data of the large measurement campaign (with approximately 112 hours of measurement and 40 000 000 RSSI values per channel) we evaluated the BER versus correlation coefficient. Therefore, we calculated the blockwise correlation as well as the corresponding blockwise BER and sorted those by correlation value. Further, we calculated the BER distribution for the following subgroups: [0 : 0.05, 0.05 : 0.1, ..., 0.95 : 1].

Figure 6.25,6.26, and 6.27 display the distribution of the blockwise BER vs. Pearson correlation coefficient of the preliminary key material from real channels. For better comparison we also plotted the simulation results from the previous section.
The results show that the BER distributions of the real-world measurements are almost always similar to the pattern of the simulation. Stronger differences are given for the scheme of Tope et al. ^{[2]}, Mathur et al. ^{[3]}, Hamida et al. ^{[4]} and Ambekar et al. ^{[5]}. Note that increasing the number of evaluated blocks may further improve the results.
For the scheme of Tope et al. ^{[6]} the BER results differ between real-world measurements and simulation results. Between the correlation coefficients of 0.65 and 1 the real-world BER does not follow the curve of the simulation results. The BER is always lower and, therefore, provides a more robust behavior occurs for the legitimate parties. The reason for that behavior might come from the specific envelope selection mechanism of the quantization scheme, which is designed for Gaussian random processes. Note, that the simulated profiles are Rayleigh distributed, while the real-world measurements are only Rayleigh-like.
The real-world based results of Mathur et al.’s ^{[7]} scheme are even better for high correlations
(> 0.75) than for simulation based channel profiles. Interestingly, the simulated curve has a regressive progression, while the curve based on real-world measurements has a degressive progression. Mathur et al.’s scheme seems to have the best property with its abrupt change of BER at a high correlation as it means that an adversary with lower correlation will not learn any information while both parties gather almost the same information if they have relatively
correlating channel measurements. However, due to public information exchange during quantization an adversary might be able to access or manipulate information to compromise the scheme.
The scheme of Hamida et al. ^{[8]} provides worst results on the real-world evaluation. The results show that the scheme is not usable. The best BER results are 0.2 which is too high to perform an efficient key extraction.
The scheme introduced by Ambekar et al. ^{[9]} provides slightly better results using real-world channels. However, the average BER at a zero correlation does still exceed the 0.3 mark.
The practice-oriented results of the schemes using Lloyd-max, mean threshold, and median threshold do not provide significant differences to the simulation results.