Backing up to Google Cloud Storage with Duplicity and Service Accounts

I wanted to use duplicity to backup to Google Cloud Storage. I looked into it briefly and found that the boto library, originally for AWS, also supports GCS, but only using authorization tokens. I’d rather use a service account, for which authorization tokens are not available.

I looked into the options and the best information I could find was a Medium post, but it also describes using authorization tokens and creating a separate GMail/Google Apps account for the access. I’d really prefer to go with a service account to avoid having to sign up another account, and to be able to use more granular ACLs for the service account.

It turns out there’s a boto plugin for GCS with OAuth2 support, but enabling a boto plugin in duplicity isn’t straight-forward. You can point it to a “plugin directory” that causes duplicity to import any python files in the directory, but this doesn’t work if you point it directly to the gcs_oauth2_boto_plugin directory.

Install Requirements

Install the following:

Create your GCS Bucket

Create a GCS bucket. In my case, I set the default storage class to “nearline” because I expect backups to be infrequently accessed (I hope), and I plan to retain the data for the minimum 30 day retention. It’s also cheaper than standard storage, so a great combination for backups.

GCS Bucket Setup

Create your service account

Next up, you need to create a service account and grant it the appropriate permissions on the bucket. Go through IAM > Service Account and create a new service account. You don’t need to grant it any roles at this time, but at the end, you should select to “Create key” and download a JSON-formatted service account key.

Go back to the bucket you created, and go to the Permissions tab. Add the service account you just created as a “Storage Object Creator” and a “Storage Object Viewer”.

Create the Boto Configuration

For this, you’ll need the Google Cloud SDK tool gsutil. Run gsutil -e -o <path to your new config>, and provide the JSON file when prompted. Note that the JSON file is only referenced by the config, so if you move it somewhere else, you’ll need to update the configuration. (Or move it first, then run it.)

This will create the necessary configuration for boto to authenticate to GCS. You’ll still need to add the support for OAuth2 authentication, so first create an empty directory to serve as your plugin directory. In my case, I created a directory ~/.config/boto/plugins for all my plugins. In it, I created one file called gcs.py whose only contents is the following:

1
import gcs_oauth2_boto_plugin

I then added the following to the bottom of my boto configuration file:

1
2
[Plugin]
plugin_directory = /home/matir/.config/boto/plugins

This will result in boto loading the gcs_oauth2_boto_plugin python module for OAuth2 authentication on GCS when being loaded into duplicity.

Setup the Duplicity Command

At this point, it’s almost like running any duplicity backup. If you chose to place your boto configuration in a non-standard location, just set the environment variable BOTO_CONFIG to point to the configuration file. I run the following:

1
2
3
4
5
6
7
export BOTO_CONFIG=${HOME}/.config/boto/boto_backups

duplicity \
  incremental \
  --full-if-older-than 30D \
  ${HOME} \
  gs://demo-backup-bucket

Hacker Summer Camp 2019: The DEF CON Data Duplication Village

One last post from Summer Camp this year (it’s been a busy month!) – this one about the “Data Duplication Village” at DEF CON. In addition to talks, the Data Duplication Village offers an opportunity to get your hands on the highest quality hacker bits – that is, copies of somewhere between 15 and 18TB of data spread across 3 6TB hard drives.

I’d been curious about the DDV for a couple of years, but never participated before. I decided to change that when I saw 6TB Ironwolf NAS drives on sale a few weeks before DEF CON. I wasn’t quite sure what to expect, as the description provided by the DDV is a little bit sparse:

6TB drive 1-3: All past convention videos that DT can find - essentially a clone of infocon.org - building on last year’s collection and re-squished with brand new codecs for your size constraining pleasures.

6TB drive 2-3: freerainbowtables hash tables (lanman, mysqlsha1, NTLM) and word lists (1-2)

6TB drive 3-3: freerainbowtables GSM A5/1, md5 hash tables, and software (2-2)

Drive 1-3 seems pretty straightforward, but I spent a lot of time debating if the other two were worth getting. (And, to be honest, I think they’re cool to have, but not sure if I’ll really make good use of them.)

I want to thank the operators of the DDV for their efforts, and also my wife for dropping off and picking up my drives while I was otherwise occupied (work obligations).

It’s worth noting that, as far as I can tell, all of the contents of the drives here is available as a torrent, so you can always get the data that way. On the other hand, torrenting 15.07 TiB (16189363384 KiB to be precise) might not be your cup of tea, especially if you have a mere 75 Mbps internet connection like mine.

If you want a detailed list of the contents of each drive (along with sha256sums), I’ve posted them to Github. If you choose to participate next year, note that your drives must be 7200 RPM SATA drives (apparently several people had to be turned away due to 5400 RPM drives, which slow down the entire cloning process).

Drive 1

Drive 1 really does seem to be a copy of infocon.org, it’s got dozens of conferences archived on it, adding up to a total of 132,253 files. Just to give you a taste, here’s a high-level index:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
./cons
./cons/2600
./cons/44Con
./cons/ACK Security Conference
./cons/ACoD
./cons/AIDE
./cons/ANYCon
./cons/ATT&CKcon
./cons/AVTokyo
./cons/Android Security Symposium
./cons/ArchC0N
./cons/Area41
./cons/AthCon
./cons/AtlSecCon
./cons/AusCERT
./cons/BalCCon
./cons/Black Alps
./cons/Black Hat
./cons/BloomCON
./cons/Blue Hat
./cons/BodyHacking
./cons/Bornhack
./cons/BotConf
./cons/BrrCon
./cons/BruCON
./cons/CERIAS
./cons/CODE BLUE
./cons/COIS
./cons/CONFidence
./cons/COUNTERMEASURE
./cons/CYBERWARCON
./cons/CackalackyCon
./cons/CactusCon
./cons/CarolinaCon
./cons/Chaos Computer Club - Camp
./cons/Chaos Computer Club - Congress
./cons/Chaos Computer Club - CryptoCon
./cons/Chaos Computer Club - Easterhegg
./cons/Chaos Computer Club - SigInt
./cons/CharruaCon
./cons/CircleCityCon
./cons/ConVerge
./cons/CornCon
./cons/CrikeyCon
./cons/CyCon
./cons/CypherCon
./cons/DEF CON
./cons/DakotaCon
./cons/DeepSec
./cons/DefCamp
./cons/DerbyCon
./cons/DevSecCon
./cons/Disobey
./cons/DojoCon
./cons/DragonJAR
./cons/Ekoparty
./cons/Electromagnetic Field
./cons/FOSDEM
./cons/FSec
./cons/GreHack
./cons/GrrCON
./cons/HCPP
./cons/HITCON
./cons/Hack In Paris
./cons/Hack In The Box
./cons/Hack In The Random
./cons/Hack.lu
./cons/Hack3rcon
./cons/HackInBo
./cons/HackWest
./cons/Hackaday
./cons/Hacker Hotel
./cons/Hackers 2 Hackers Conference
./cons/Hackers At Large
./cons/Hackfest
./cons/Hacking At Random
./cons/Hackito Ergo Sum
./cons/Hacks In Taiwan
./cons/Hacktivity
./cons/Hash Days
./cons/HouSecCon
./cons/ICANN
./cons/IEEE Security and Privacy
./cons/IETF
./cons/IRISSCERT
./cons/Infiltrate
./cons/InfoWarCon
./cons/Insomnihack
./cons/KazHackStan
./cons/KiwiCon
./cons/LASCON
./cons/LASER
./cons/LangSec
./cons/LayerOne
./cons/LevelUp
./cons/LocoMocoSec
./cons/Louisville Metro InfoSec
./cons/MISP Summit
./cons/NANOG
./cons/NoNameCon
./cons/NolaCon
./cons/NorthSec
./cons/NotACon
./cons/NotPinkCon
./cons/Nuit Du Hack
./cons/NullCon
./cons/O'Reilly Security
./cons/OISF
./cons/OPCDE
./cons/OURSA
./cons/OWASP
./cons/Observe Hack Make
./cons/OffensiveCon
./cons/OzSecCon
./cons/PETS
./cons/PH-Neutral
./cons/Pacific Hackers
./cons/PasswordsCon
./cons/PhreakNIC
./cons/Positive Hack Days
./cons/Privacy Camp
./cons/QuahogCon
./cons/REcon
./cons/ROMHACK
./cons/RSA
./cons/RVAsec
./cons/Real World Crypto
./cons/RightsCon
./cons/RoadSec
./cons/Rooted CON
./cons/Rubicon
./cons/RuhrSec
./cons/RuxCon
./cons/S4
./cons/SANS
./cons/SEC-T
./cons/SHA2017
./cons/SIRAcon
./cons/SOURCE
./cons/SaintCon
./cons/SecTor
./cons/SecureWV
./cons/Securi-Tay
./cons/Security BSides
./cons/Security Fest
./cons/Security Onion
./cons/Security PWNing
./cons/Shakacon
./cons/ShellCon
./cons/ShmooCon
./cons/ShowMeCon
./cons/SkyDogCon
./cons/SteelCon
./cons/SummerCon
./cons/SyScan
./cons/THREAT CON
./cons/TROOPERS
./cons/TakeDownCon
./cons/Texas Cyber Summit
./cons/TheIACR
./cons/TheLongCon
./cons/TheSAS
./cons/Thotcon
./cons/Toorcon
./cons/TrustyCon
./cons/USENIX ATC
./cons/USENIX Enigma
./cons/USENIX Security
./cons/USENIX WOOT
./cons/Unrestcon
./cons/Virus Bulletin
./cons/WAHCKon
./cons/What The Hack
./cons/Wild West Hackin Fest
./cons/You Shot The Sheriff
./cons/Zero Day Con
./cons/ZeroNights
./cons/c0c0n
./cons/eth0
./cons/hardware.io
./cons/outerz0ne
./cons/r00tz Asylum
./cons/r2con
./cons/rootc0n
./cons/t2 infosec
./cons/x33fcon
./documentaries
./documentaries/Hacker Movies
./documentaries/Hacking Documentaries
./documentaries/Other
./documentaries/Pirate Documentary
./documentaries/Tech Documentary
./documentaries/Tools
./infocon.jpg
./mirrors
./mirrors/cryptome.org-July-2019.rar
./mirrors/gutenberg-15-July-2019.net.au.rar
./rainbow tables
./rainbow tables/## READ ME RAINBOW TABLES ##.txt
./rainbow tables/rainbow table software
./skills
./skills/Lock Picking
./skills/MAKE

Drive 2

Drive 2 contains the promised rainbow tables (lanman, ntlm, and mysqlsha1) as well as a bunch of wordlists. I actually wonder how a 128GB wordlist would compare to applying rules to something like rockyou – bigger is not always better, and often, you want high yield unless you’re trying to crack something obscure.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
./lanman
./lanman/lm_all-space#1-7_0
./lanman/lm_all-space#1-7_1
./lanman/lm_all-space#1-7_2
./lanman/lm_all-space#1-7_3
./lanman/lm_lm-frt-cp437-850#1-7_0
./lanman/lm_lm-frt-cp437-850#1-7_1
./lanman/lm_lm-frt-cp437-850#1-7_2
./lanman/lm_lm-frt-cp437-850#1-7_3
./mysqlsha1
./mysqlsha1/mysqlsha1_loweralpha#1-10_0
./mysqlsha1/mysqlsha1_loweralpha#1-10_1
./mysqlsha1/mysqlsha1_loweralpha#1-10_2
./mysqlsha1/mysqlsha1_loweralpha#1-10_3
./mysqlsha1/mysqlsha1_loweralpha-numeric#1-10_0
./mysqlsha1/mysqlsha1_loweralpha-numeric#1-10_16
./mysqlsha1/mysqlsha1_loweralpha-numeric#1-10_24
./mysqlsha1/mysqlsha1_loweralpha-numeric#1-10_8
./mysqlsha1/mysqlsha1_loweralpha-numeric-space#1-8_0
./mysqlsha1/mysqlsha1_loweralpha-numeric-space#1-8_1
./mysqlsha1/mysqlsha1_loweralpha-numeric-space#1-8_2
./mysqlsha1/mysqlsha1_loweralpha-numeric-space#1-8_3
./mysqlsha1/mysqlsha1_loweralpha-numeric-space#1-9_0
./mysqlsha1/mysqlsha1_loweralpha-numeric-space#1-9_1
./mysqlsha1/mysqlsha1_loweralpha-numeric-space#1-9_2
./mysqlsha1/mysqlsha1_loweralpha-numeric-space#1-9_3
./mysqlsha1/mysqlsha1_loweralpha-numeric-symbol32-space#1-7_0
./mysqlsha1/mysqlsha1_loweralpha-numeric-symbol32-space#1-7_1
./mysqlsha1/mysqlsha1_loweralpha-numeric-symbol32-space#1-7_2
./mysqlsha1/mysqlsha1_loweralpha-numeric-symbol32-space#1-7_3
./mysqlsha1/mysqlsha1_loweralpha-numeric-symbol32-space#1-8_0
./mysqlsha1/mysqlsha1_loweralpha-numeric-symbol32-space#1-8_1
./mysqlsha1/mysqlsha1_loweralpha-numeric-symbol32-space#1-8_2
./mysqlsha1/mysqlsha1_loweralpha-numeric-symbol32-space#1-8_3
./mysqlsha1/mysqlsha1_loweralpha-space#1-9_0
./mysqlsha1/mysqlsha1_loweralpha-space#1-9_1
./mysqlsha1/mysqlsha1_loweralpha-space#1-9_2
./mysqlsha1/mysqlsha1_loweralpha-space#1-9_3
./mysqlsha1/mysqlsha1_mixalpha-numeric-symbol32-space#1-7_0
./mysqlsha1/mysqlsha1_mixalpha-numeric-symbol32-space#1-7_1
./mysqlsha1/mysqlsha1_mixalpha-numeric-symbol32-space#1-7_2
./mysqlsha1/mysqlsha1_mixalpha-numeric-symbol32-space#1-7_3
./mysqlsha1/mysqlsha1_numeric#1-12_0
./mysqlsha1/mysqlsha1_numeric#1-12_1
./mysqlsha1/mysqlsha1_numeric#1-12_2
./mysqlsha1/mysqlsha1_numeric#1-12_3
./mysqlsha1/rainbow table software
./ntlm
./ntlm/ntlm_alpha-space#1-9_0
./ntlm/ntlm_alpha-space#1-9_1
./ntlm/ntlm_alpha-space#1-9_2
./ntlm/ntlm_alpha-space#1-9_3
./ntlm/ntlm_hybrid2(alpha#1-1,loweralpha#5-5,loweralpha-numeric#2-2,numeric#1-3)#0-0_0
./ntlm/ntlm_hybrid2(alpha#1-1,loweralpha#5-5,loweralpha-numeric#2-2,numeric#1-3)#0-0_1
./ntlm/ntlm_hybrid2(alpha#1-1,loweralpha#5-5,loweralpha-numeric#2-2,numeric#1-3)#0-0_2
./ntlm/ntlm_hybrid2(alpha#1-1,loweralpha#5-5,loweralpha-numeric#2-2,numeric#1-3)#0-0_3
./ntlm/ntlm_hybrid2(loweralpha#7-7,numeric#1-3)#0-0_0
./ntlm/ntlm_hybrid2(loweralpha#7-7,numeric#1-3)#0-0_1
./ntlm/ntlm_hybrid2(loweralpha#7-7,numeric#1-3)#0-0_2
./ntlm/ntlm_hybrid2(loweralpha#7-7,numeric#1-3)#0-0_3
./ntlm/ntlm_loweralpha-numeric#1-10_0
./ntlm/ntlm_loweralpha-numeric#1-10_16
./ntlm/ntlm_loweralpha-numeric#1-10_24
./ntlm/ntlm_loweralpha-numeric#1-10_8
./ntlm/ntlm_loweralpha-numeric-space#1-8_0
./ntlm/ntlm_loweralpha-numeric-space#1-8_1
./ntlm/ntlm_loweralpha-numeric-space#1-8_2
./ntlm/ntlm_loweralpha-numeric-space#1-8_3
./ntlm/ntlm_loweralpha-numeric-symbol32-space#1-7_0
./ntlm/ntlm_loweralpha-numeric-symbol32-space#1-7_1
./ntlm/ntlm_loweralpha-numeric-symbol32-space#1-7_2
./ntlm/ntlm_loweralpha-numeric-symbol32-space#1-7_3
./ntlm/ntlm_loweralpha-numeric-symbol32-space#1-8_0
./ntlm/ntlm_loweralpha-numeric-symbol32-space#1-8_1
./ntlm/ntlm_loweralpha-numeric-symbol32-space#1-8_2
./ntlm/ntlm_loweralpha-numeric-symbol32-space#1-8_3
./ntlm/ntlm_loweralpha-space#1-9_0
./ntlm/ntlm_loweralpha-space#1-9_1
./ntlm/ntlm_loweralpha-space#1-9_2
./ntlm/ntlm_loweralpha-space#1-9_3
./ntlm/ntlm_mixalpha-numeric#1-8_0
./ntlm/ntlm_mixalpha-numeric#1-8_1
./ntlm/ntlm_mixalpha-numeric#1-8_2
./ntlm/ntlm_mixalpha-numeric#1-8_3
./ntlm/ntlm_mixalpha-numeric#1-9_0
./ntlm/ntlm_mixalpha-numeric#1-9_16
./ntlm/ntlm_mixalpha-numeric#1-9_32
./ntlm/ntlm_mixalpha-numeric#1-9_48
./ntlm/ntlm_mixalpha-numeric-all-space#1-7_0
./ntlm/ntlm_mixalpha-numeric-all-space#1-7_1
./ntlm/ntlm_mixalpha-numeric-all-space#1-7_2
./ntlm/ntlm_mixalpha-numeric-all-space#1-7_3
./ntlm/ntlm_mixalpha-numeric-all-space#1-8_0
./ntlm/ntlm_mixalpha-numeric-all-space#1-8_16
./ntlm/ntlm_mixalpha-numeric-all-space#1-8_24
./ntlm/ntlm_mixalpha-numeric-all-space#1-8_32
./ntlm/ntlm_mixalpha-numeric-all-space#1-8_8
./ntlm/ntlm_mixalpha-numeric-space#1-7_0
./ntlm/ntlm_mixalpha-numeric-space#1-7_1
./ntlm/ntlm_mixalpha-numeric-space#1-7_2
./ntlm/ntlm_mixalpha-numeric-space#1-7_3
./ntlm/rainbow table software
./rainbow table software
./rainbow table software/Free Rainbow Tables » Distributed Rainbow Table Generation » LM, NTLM, MD5, SHA1, HALFLMCHALL, MSCACHE.mht
./rainbow table software/converti2_0.3_src.7z
./rainbow table software/converti2_0.3_win32_mingw.7z
./rainbow table software/converti2_0.3_win32_vc.7z
./rainbow table software/converti2_0.3_win64_mingw.7z
./rainbow table software/converti2_0.3_win64_vc.7z
./rainbow table software/rcracki_mt_0.7.0_linux_x86_64.7z
./rainbow table software/rcracki_mt_0.7.0_src.7z
./rainbow table software/rcracki_mt_0.7.0_win32_mingw.7z
./rainbow table software/rcracki_mt_0.7.0_win32_vc.7z
./rainbow table software/rti2formatspec.pdf
./rainbow table software/rti2rto_0.3_beta2_win32_vc.7z
./rainbow table software/rti2rto_0.3_beta2_win64_vc.7z
./rainbow table software/rti2rto_0.3_src.7z
./rainbow table software/rti2rto_0.3_win32_mingw.7z
./rainbow table software/rti2rto_0.3_win64_mingw.7z
./word lists
./word lists/SecLists-master.rar
./word lists/WPA-PSK WORDLIST 3 Final (13 GB).rar
./word lists/Word Lists archive - infocon.org.torrent
./word lists/crackstation-human-only.txt.rar
./word lists/crackstation.realuniq.rar
./word lists/fbnames.rar
./word lists/human0id word lists.rar
./word lists/openlibrary_wordlist.rar
./word lists/pwgen.rar
./word lists/pwned-passwords-2.0.txt.rar
./word lists/pwned-passwords-ordered-2.0.rar
./word lists/xsukax 128GB word list all 2017 Oct.7z

Drive 3

Drive 3 contains more rainbow tables, this time for A5-1 (GSM encryption), and extensive tables for MD5. It appears to contain the same software and wordlists as Drive 2.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
./A51
./A51 rainbow tables - infocon.org.torrent
./A51/Decoding-Gsm.pdf
./A51/a51_table_100.dlt
./A51/a51_table_108.dlt
./A51/a51_table_116.dlt
./A51/a51_table_124.dlt
./A51/a51_table_132.dlt
./A51/a51_table_140.dlt
./A51/a51_table_148.dlt
./A51/a51_table_156.dlt
./A51/a51_table_164.dlt
./A51/a51_table_172.dlt
./A51/a51_table_180.dlt
./A51/a51_table_188.dlt
./A51/a51_table_196.dlt
./A51/a51_table_204.dlt
./A51/a51_table_212.dlt
./A51/a51_table_220.dlt
./A51/a51_table_230.dlt
./A51/a51_table_238.dlt
./A51/a51_table_250.dlt
./A51/a51_table_260.dlt
./A51/a51_table_268.dlt
./A51/a51_table_276.dlt
./A51/a51_table_292.dlt
./A51/a51_table_324.dlt
./A51/a51_table_332.dlt
./A51/a51_table_340.dlt
./A51/a51_table_348.dlt
./A51/a51_table_356.dlt
./A51/a51_table_364.dlt
./A51/a51_table_372.dlt
./A51/a51_table_380.dlt
./A51/a51_table_388.dlt
./A51/a51_table_396.dlt
./A51/a51_table_404.dlt
./A51/a51_table_412.dlt
./A51/a51_table_420.dlt
./A51/a51_table_428.dlt
./A51/a51_table_436.dlt
./A51/a51_table_492.dlt
./A51/a51_table_500.dlt
./A51/rainbow table software
./LANMAN rainbow tables - infocon.org.torrent
./MD5 rainbow tables - infocon.org.torrent
./MySQL SHA-1 rainbow tables - infocon.org.torrent
./NTLM rainbow tables - infocon.org.torrent
./md5
./md5/md5_alpha-space#1-9_0
./md5/md5_alpha-space#1-9_1
./md5/md5_alpha-space#1-9_2
./md5/md5_alpha-space#1-9_3
./md5/md5_hybrid2(loweralpha#7-7,numeric#1-3)#0-0_0
./md5/md5_hybrid2(loweralpha#7-7,numeric#1-3)#0-0_1
./md5/md5_hybrid2(loweralpha#7-7,numeric#1-3)#0-0_2
./md5/md5_hybrid2(loweralpha#7-7,numeric#1-3)#0-0_3
./md5/md5_loweralpha#1-10_0
./md5/md5_loweralpha#1-10_1
./md5/md5_loweralpha#1-10_2
./md5/md5_loweralpha#1-10_3
./md5/md5_loweralpha-numeric#1-10_0
./md5/md5_loweralpha-numeric#1-10_16
./md5/md5_loweralpha-numeric#1-10_24
./md5/md5_loweralpha-numeric#1-10_8
./md5/md5_loweralpha-numeric-space#1-8_0
./md5/md5_loweralpha-numeric-space#1-8_1
./md5/md5_loweralpha-numeric-space#1-8_2
./md5/md5_loweralpha-numeric-space#1-8_3
./md5/md5_loweralpha-numeric-space#1-9_0
./md5/md5_loweralpha-numeric-space#1-9_1
./md5/md5_loweralpha-numeric-space#1-9_2
./md5/md5_loweralpha-numeric-space#1-9_3
./md5/md5_loweralpha-numeric-symbol32-space#1-7_0
./md5/md5_loweralpha-numeric-symbol32-space#1-7_1
./md5/md5_loweralpha-numeric-symbol32-space#1-7_2
./md5/md5_loweralpha-numeric-symbol32-space#1-7_3
./md5/md5_loweralpha-numeric-symbol32-space#1-8_0
./md5/md5_loweralpha-numeric-symbol32-space#1-8_1
./md5/md5_loweralpha-numeric-symbol32-space#1-8_2
./md5/md5_loweralpha-numeric-symbol32-space#1-8_3
./md5/md5_loweralpha-space#1-9_0
./md5/md5_loweralpha-space#1-9_1
./md5/md5_loweralpha-space#1-9_2
./md5/md5_loweralpha-space#1-9_3
./md5/md5_mixalpha-numeric#1-9_0
./md5/md5_mixalpha-numeric#1-9_0-complete
./md5/md5_mixalpha-numeric#1-9_16
./md5/md5_mixalpha-numeric#1-9_32
./md5/md5_mixalpha-numeric#1-9_48
./md5/md5_mixalpha-numeric-all-space#1-7_0
./md5/md5_mixalpha-numeric-all-space#1-7_1
./md5/md5_mixalpha-numeric-all-space#1-7_2
./md5/md5_mixalpha-numeric-all-space#1-7_3
./md5/md5_mixalpha-numeric-all-space#1-8_0
./md5/md5_mixalpha-numeric-all-space#1-8_16
./md5/md5_mixalpha-numeric-all-space#1-8_24
./md5/md5_mixalpha-numeric-all-space#1-8_32
./md5/md5_mixalpha-numeric-all-space#1-8_8
./md5/md5_mixalpha-numeric-space#1-7_0
./md5/md5_mixalpha-numeric-space#1-7_1
./md5/md5_mixalpha-numeric-space#1-7_2
./md5/md5_mixalpha-numeric-space#1-7_3
./md5/md5_mixalpha-numeric-space#1-8_0
./md5/md5_mixalpha-numeric-space#1-8_1
./md5/md5_mixalpha-numeric-space#1-8_2
./md5/md5_mixalpha-numeric-space#1-8_3
./md5/md5_numeric#1-14_0
./md5/md5_numeric#1-14_1
./md5/md5_numeric#1-14_2
./md5/md5_numeric#1-14_3
./rainbow table software
./rainbow table software/Free Rainbow Tables » Distributed Rainbow Table Generation » LM, NTLM, MD5, SHA1, HALFLMCHALL, MSCACHE.mht
./rainbow table software/converti2_0.3_src.7z
./rainbow table software/converti2_0.3_win32_mingw.7z
./rainbow table software/converti2_0.3_win32_vc.7z
./rainbow table software/converti2_0.3_win64_mingw.7z
./rainbow table software/converti2_0.3_win64_vc.7z
./rainbow table software/rcracki_mt_0.7.0_linux_x86_64.7z
./rainbow table software/rcracki_mt_0.7.0_src.7z
./rainbow table software/rcracki_mt_0.7.0_win32_mingw.7z
./rainbow table software/rcracki_mt_0.7.0_win32_vc.7z
./rainbow table software/rti2formatspec.pdf
./rainbow table software/rti2rto_0.3_beta2_win32_vc.7z
./rainbow table software/rti2rto_0.3_beta2_win64_vc.7z
./rainbow table software/rti2rto_0.3_src.7z
./rainbow table software/rti2rto_0.3_win32_mingw.7z
./rainbow table software/rti2rto_0.3_win64_mingw.7z
./word lists
./word lists/SecLists-master.rar
./word lists/WPA-PSK WORDLIST 3 Final (13 GB).rar
./word lists/Word Lists archive - infocon.org.torrent
./word lists/crackstation-human-only.txt.rar
./word lists/crackstation.realuniq.rar
./word lists/fbnames.rar
./word lists/human0id word lists.rar
./word lists/openlibrary_wordlist.rar
./word lists/pwgen.rar
./word lists/pwned-passwords-2.0.txt.rar
./word lists/pwned-passwords-ordered-2.0.rar
./word lists/xsukax 128GB word list all 2017 Oct.7z

CVE-2019-10071: Timing Attack in HMAC Verification in Apache Tapestry

Description

Apache Tapestry uses HMACs to verify the integrity of objects stored on the client side. This was added to address the Java deserialization vulnerability disclosed in CVE-2014-1972. In the fix for the previous vulnerability, the HMACs were compared by string comparison, which is known to be vulnerable to timing attacks.

Affected Versions

  • Apache Tapestry 5.3.6 through current releases.

Mitigation

No new release of Tapestry has occurred since the issue was reported. Affected organizations may want to consider locally applying commit d3928ad44714b949d247af2652c84dae3c27e1b1.

Timeline

  • 2019-03-12: Issue discovered.
  • 2019-03-13: Issue reported to security@apache.org.
  • 2019-03-29: Pinged thread to ask for update.
  • 2019-04-19: Fix committed.
  • 2019-04-23: Asked about release timeline, response “in the upcoming months”
  • 2019-05-28: Pinging again about release.
  • 2019-06-24: Asked again, asked for CVE number assigned. No update on timeline.
  • 2019-08-22: Disclosure posted.

Credit

This vulnerability was discovered by David Tomaschik of the Google Security Team.


Hacker Summer Camp 2019: CTFs for Fun & Profit

Okay, I’m back from Summer Camp and have caught up (slightly) on life. I had the privilege of giving a talk at BSidesLV entitled “CTFs for Fun and Profit: Playing Games to Build Your Skills.” I wanted to post a quick link to my slides and talk about the IoT CTF I had the chance to play.

I played in the IoT Village CTF at DEF CON, which was interesting because it uses real-world devices with real-world vulnerabilities instead of the typical made-up challenges in a CTF. On the other hand, I’m a little disappointed that it seems pretty similar (maybe even the same) year-to-year, not providing much variety or new learning experiences if you’ve played before.


Hacker Summer Camp 2019: What I'm Bringing & Protecting Yourself

I’ve begun to think about what I’ll take to Hacker Summer Camp this year, and I thought I’d share some of it as part of my Hacker Summer Camp blog post series. I hope it will be useful to veterans, but particularly to first timers who might have no idea what to expect – as that’s how I felt my first time.

Since it’s gotten so close, I’ll also talk about what steps you should take to protect yourself.

Packing

General Packing

I won’t state the obvious in terms of packing most of your basic needs, including clothing and toiletries, but I will remind you that Las Vegas will be super hot. Bring clothes for hot days, and pack deodorant! Keep in mind that some of the clubs have a dress code, so if that’s your thing, you’ll want to bring clubbing clothes. (The dress code tends not to be too high, but often pants and a collared shirt.)

I will suggest bringing a reusable water bottle to help cope with the heat. Just before last summer camp, I bought a Simple Modern vacuum insulated bottle, and I absolutely love it. I’ll bring it again this year to stay hydrated. Because I hate heat, I’ll also be bringing a cooling towel, which is surprisingly effective at cooling me off. Perhaps it’s a placebo effect, but I’ll take it.

Remember that large parts of DEF CON are cash only, so you’ll need to bring cash (obviously). At least $300 for a badge, plus more for swag, bars, etc. ATMs on the casino floors are probably safe to use, but will still charge you fairly hefty fees.

Tech Gear

There’s two schools of thought on bringing tech gear: minimalist and kitchen sink. I happen to be in the kitchen sink side of things. I’ll be bringing my laptop and about a whole bunch of accessories. In fact, I have a whole travel kit that I’ll detail in a future post, but a few highlights include:

On the other hand, some people want the disconnected experience and bring little to no tech. Sometimes this is because of concerns over “being hacked”, but sometimes this is just to focus on the face-to-face time.

Shipping

There are some consumables where I just find it easier to ship to my hotel. Note that the hotel will charge you for receiving a package, but I still find it cheaper/easier to have these things delivered directly.

Getting a case of water delivered is much cheaper than buying from the hotel gift shop. Another option is to hit up a CVS or Walgreens on the strip for some bottled water.

I’m a bit of a Red Bull addict, so I often get a few packs delivered to have on hand. The Red Bull Red Edition is a nice twist on the classic that’s worth a try if you haven’t had the pleasure.

Safety & Security

DEF CON has a reputation for being the “most dangerous network in the world”, but I think this is completely overblown. It defies logic that an attacker with a 0-day on a modern operating system would use it to perform untargeted attacks at DEF CON. If their traffic is captured, they’ve burned their 0-day, and probably to grab some random attendees data – it’s just not worth it to them.

That being said, you shouldn’t make yourself a target either. There are some simple steps you can (and should) take to protect yourself:

  • Use a VPN service for your traffic. I like Private Internet Access for a commercial provider.
  • Don’t connect to open WiFi networks.
  • Don’t accept certificate errors.
  • Don’t plug your phone into strange USB plugs.
  • Use HTTPS

These are all simple steps to protect yourself, both at DEF CON, and in general. You really ought to observe them all the time – the internet is a dangerous place in general!

To be honest, I worry more about physical security in Las Vegas – don’t carry too much cash, keep your wits about you, and watch your belongings. Use the in-room safe (they’re not perfect, but they’re better than nothing) to protect your goods.

Be aware of hotel policies on entering rooms – ever since the Las Vegas shooting, they’ve become much more invasive with forcing their way into hotel rooms. I recommend keeping anything valuable locked up and out of sight, and be aware of potential impostors using the pretext of being a hotel employee.

Good luck, and have fun in just over a week!