DEFCON30 Car Hacking Village CTF Memorandum
~Falling from 1st to 4th Place ~
Hello. We have joined the Car Hacking Village (CHV) CTF of DEFCON30 as team tyt4001. Unfortunately, we ended in 4th place, though as in the title we were originally 1st place according to the “rules” that were announced. There were so much back-and-forth until the final places were announced. In this article, we will like to share what happened as a comment to the CTF organizers as well as an alert for people considering to participate in future CTFs.
Disclaimer; The translator’s mother tongue is not English – if you understand what we’re trying to say that’s good enough.
1. CTF Style
CHV CTF of DEFCON30 was a Hybrid style – some challenges had to be solved on-site while others could have been from remote. We had 4 members from our company, and 2 members from our client. Within the 4 members from our company, we had 2 people visiting Las Vegas, and attending the CTF. Those who was on-site were mainly in charge of handling the on-site challenges and interacting with the organizers while other members were mainly solving challenges that could be solved remotely.
2. On-site Challenges
The village had a few ECUs and items that will actually be utilized in real-world cars, and challenges were given. Mainly the challenges required the challengers to access the devices via PC. Some challenges required Bluetooth connection to devices.
All of the on-site challenges were intended to be solved at the village. However, we had a RaspberryPi that could be used as a backdoor to the village, so remote members could access once the RaspberryPi was set.
3. Challenges Available Remotely
One of the hottest themes for this year’s CTF was those related to CloudCar. There were challenges available remotely in earlier DEF CONs, though CloudCar was new. The virtual car would continuosly run a specific course, and challengers will access to the car using SSH and analyze ECUs or the car’s behavior for points. On top of that, laps were counted as points.
| Lap Points: In order to encourage a healthy and functional virtual car, each lap around the virtual track is worth either 1 point (on Friday through Saturday at 10 AM), 2 Points (on Saturday after 10 AM through Sunday at 10 AM), and 5 points (Sunday from 10 AM to 12 PM). |
As written in the rules, the laps were a crucial part on winning the CTF as you would get points accordingly to the day of the CTF. Also, other challenges related to CloudCar were valued high points. Therefore, we have concluded that we will assign higher portion of our resources to those challenges.
The following diagram shows how much points the CloudCar represented at minimum in the CTF. Additionaly, lap points were to be added so it is easy to figure that CloudCar was the “thing” to win the CTF. Also, please remember about “Red Balloon”, which initially has a higher portion compared to CloudCar without the laps. We will touch Red Balloon a bit later.
4. CloudCar
In order to make the CloudCar run, you need Gas O’lean (gasoline). Also, since the front glass will get dirty, you need Lemmon Ade (washer fluid) too. You needed to have Gas O’lean cards and Lemmon Ade cards to fuel them accordingly, and it was announced that the organizers (staff) of the CTF had these, and some were placed within the venue (i.e., across other villages). In the end, you had to buy coffee, or have something to exchange, or even win push-ups to gain these cards. We had spent a vast amount of time to gain these cards – and ended up with 7 Gas O’lean cards with 11 Lemmon Ade cards. We believe we have obtained more cards than most of the challengers. Even though it was announced that other village staff members will have these cards, none knew about its existent when we ran for it.
As we analyzed CloudCar, we realized that by sending specific CAN command, you could run the car without Gas O’lean. In real world you cannot run a gasoline car without gas, so we have reported this as a bug – which the staff denied. The car was meant run without Gas O’lean in certain circumstances. As a result, we were able to run CloudCar without Gas.
(*Do note that we have used all the Gas O’lean cards since running with fuel was much more stable compared to none.)
We have made the car running at 127km/h by modifying a specific CAN packet’s payload in the first day. By combining these methods we reached 123 laps within the first day.
However, after the end of 1st day, the organizers gave the following tweet. From the tweet you may want to interpret it as the laps were counted before the tweet came out, though the laps stopped counting and our 123 laps were gone (and laps were reset continuously after the tweet).
The next day, the organizer tweeted that laps will be counted again and will be disabled in the beginning of the 3rd day.
* The above tweets are shown in Japan standard time (GMT +9)
We have analyzed CloudCar further and found a way to run it in 255kim/h by sending a speed modification packet after gaining SecurityAccess in a session.
Now we want to make clear what we have agreed with the organizers to ensure we were competing the CTF in accordance with the rules. This is what we have agreed before the end of day 2.
・ Laps will have 2x scores for the 2nd day as the rule says.
・ We can run the CloudCar after the village closes in 5:30PM until the beginning of day 3, 10AM.
・ The car is meant to run without Gas O’lean (mentioned earlier).
Since the rules were clear, we have run the CloudCar all night long before 10AM of day3, resulting with 1,533 laps. Please note that the “Score” section is related to other challenges within the CloudCar excluding laps (though one team mysteriously got more than the maximum points they could – interesting).
The CloudCar had the following issues that could be taken as bugs or spec, making it get stuck in 0km/h.
- The simulator will not work properly if multiple windows were launched
- Even if the car was running “normally” it would suddenly stop and get stuck in 0km/h. No CAN command will be accepted once entering this state.
- Even if the CloudCar had Gas O’lean the car won’t start running.
- Changing the camera angle will cause the simulator to crash.
- The simulator will crash along with the browser.
These issues caused us from counting laps for a certain amount of time, though when the CloudCar was running as it should, we were able to have 2 laps while other teams counted 1 .We thought we were the only team that were able to run the car 255km/h. At the beginning of day 3 we confirmed with the organizer that running the CloudCar in 255km/h was a legitimate technique, and we were praised as the only team who were capable to find this out.
5. Discussion after the CTF
The CTF closed in 12PM, of day 3. Before adding the lap points, the place were as follows.
We were in 4th place at this point. The organizer said they wanted to talk with each of the team who placed higher than 5th one by one. When our turn came, we were told;
“Other teams could not use CloudCar full time due to serious bugs. We will give you 2x points for the laps, though we want your to agree giving half the points to other teams“
We thought this was too much – 1,533 laps was an outcome of deeply analyzing ECU, collecting cards, and running the CloudCar throughout the night. However, we had very little chance (or no) to refuse this suggestion. We said OK, because we believed that with the lap points we will finish in 1st place anyway.
5.1. Discussion – 1
Now the lap points were added, we took 1st place as expected.
Though our win didn’t last long. The team in second place got additional points, which are described in the following picture.
The upper-right was the 1,533 points that we have “agreed” to give to other teams. The left down 300 points was an additional point with no rationale given, which other teams argued too. The organizer said this was a special point for them reporting bugs. Even with the 300 points, our team will still be in 1st place, though their team even had 164 points a 2x score of their lap counts making them 1st place. Due to the 300 and 164 additional points now they took 1st prize, other teams argued since they were the only team who got additional points.
5.2. Discussion – 2
We were surprised that we fell from 1st place, though we argued if that was the case, laps from day 1 should be counted – and was accepted.
With this 123 points, we returned to the first place, though were skeptical. Why are the rules changing so much?
5.3. Discussion – 3
Other teams also argued strongly, too. Especially one team that counted more than 900 laps argued that 1,533 was not enough (since they should have more scores if counted by the rules) – we agreed to their claims.
We argued as well for the mysterious additional points. First of all, the only agreement we have made was adding 1,533 points to other teams and other additional points were a bolt from the blue. If we knew the existence of such additional points, we would have never agreed giving other teams 1,533 points.
The organizers went into a long discussion, though challengers were not informed who were involved, nor what the context was. In the end, the organizers stated that they will cancel counting lap points with the following tweet as justification.
5.4. End of Discussion – from 1st place to 4th
Finally, the organizers claimed that they will cancel adding any of the lap points. As their ground, they have claimed that “we have tweeted that the CloudCar’s environment is un-reliable. We once claimed that we will start counting, though we believed that the environment continued to be un-reliable. We want to be fair to all teams.”
The CloudCar was un-reliable for everyone – we were struggling too. As written above, the CloudCar had issues which we came from bugs/specs, we had to be creative. Here are some examples.
- The simulator will not work properly if multiple windows were launched
- We assigned one of our team members to have full attention to keep the CloudCar running.
- Even if the car was running “normally” it would suddenly stop and get stuck in 0km/h. No CAN command will be accepted once entering this state.
- Once the CloudCar got stuck, we re-sent CAN commands, even re-started the simulator.
- Even if the CloudCar had Gas O’lean the car won’t start running.
- We replayed CAN packets that we have dumped every time it stopped.
- The simulator will crash along with the browser.
- We re-started the browser per crash.
The following video shows the progress for the CloudCar to ignite the engine.
We appealed that we were also influenced by the buggy CloudCar. However, the organizers took our claim that we have accepted that the CloudCar was unreliable, and agree that therefore the lap points were gone. Our minds were quite blown, and we fell from 1st place to 4th.
By the way, “Red Balloon” challenges were gone at day 3 – the staffs in charge went home and teams were unable to answer. Some teams were disputing the matter, though nothing was done in the term of fairness. Teams that had already solved challenges were not taken their points. As noted above, Red Balloon challenges had the largest portion of points without lappoints. The smallest challenge was 250 points, with the next waying 500 points – which is huge enough to influence the challenge, though unlike the lap points, points resumed. More than one high ranking teams already solved several challenges of Red Balloon by the end of day 2, organizers did not take those points away in temrs of fairness.
6. In the End
The CloudCar was fascinating. You could attack it using UDS likewise a real car, and even remotely. The on-site challenges were massive too. We were very sorry such a CTF ended with rules changing again and again,after the competition was closed, and places rambling. There is nothing you can do as a challengers, when rules change after the competition. If the announcement of the laps were made at the beginning of day 3, we could have changed how we assigned our resources, but after the competition – it’s too late.
The challenges of the CTF were very well made, and we were literally working as one team with our partner company. It was a very valuable experience and we were really enjoying it. That is why the bad aftertaste was devasting for us. We believe that we had followed the rules, and scroed our laps. We went ahead of other teams since we spent reasonable resources and came up with creative methods that they could not. Invalidating lap points just because we were the only one who found the loophole is irrational, and a very surprising attitude for a CTF organizer. CTFs are meant to evaluate participants creativity, and if challengers come up with an unimaginable answer, organizers should welcome it. It was more than shocking that our creativity was not welcomed but denied in a DEF CON CTF.
Acknowledgements
We would like to thank team ND4 who also came from Japan. We were able to remain in 4th place since you let us use your circuit tester.
We would like to thank our partner for joining our team. We had more variety in the challenges we were capable of handling.
Finally, we would like to thank the reviewers for reviewing this memorandum before going public.
Contactお問い合わせ
お問い合わせは、プライバシーポリシーをご参照の上、
sales.team@00one.jpまでご連絡ください。