tag:blogger.com,1999:blog-14046602049664540422024-03-13T11:25:01.576-07:00Meme OverMostly game development, computer security, and information warfare. Mostly.Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.comBlogger62125tag:blogger.com,1999:blog-1404660204966454042.post-70507789263039162662023-05-26T09:38:00.005-07:002023-05-26T09:39:29.284-07:00New Avalon on a semi-hiatus<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoYKj2Y_iP759z4QMgVwBT-v-U9QVJgdYvsBHxurWeVT32re7egj1TUCbOCXoRlM5I-HRQkWR4zdWFs-9M4DHObjxnrDrgHzUPUBKsQGvmZtDh0MYApo3F7mdsV4Ot03Bbir2tnRF2diDipKUn2DH13jO44IlzkoeEvJ_AAqqpsemPDnOhlk_sHhCA/s2250/poster_logo_sm.jpg" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="2250" data-original-width="1591" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjoYKj2Y_iP759z4QMgVwBT-v-U9QVJgdYvsBHxurWeVT32re7egj1TUCbOCXoRlM5I-HRQkWR4zdWFs-9M4DHObjxnrDrgHzUPUBKsQGvmZtDh0MYApo3F7mdsV4Ot03Bbir2tnRF2diDipKUn2DH13jO44IlzkoeEvJ_AAqqpsemPDnOhlk_sHhCA/s320/poster_logo_sm.jpg" width="226" /></a></div> After roughly a year of working unsuccessfully to secure funding I'm putting New Avalon on the backburner for a while. We talked to more than 50 publishers, venture capitalists, other investors, consultants and other experts looking for a way to support development but in the current climate there's not a lot of appetite for risk, especially from a new team without an advanced prototype.<p></p><p>I'm planning on pushing the project forward in my spare time in the hopes that we can one day get the band back together and find some funding once we're a little bit further along. In the mean time I've accepted a job at Odyssey Interactive to work tech on Omega Strikers and anything else they cook up, I did some contract work for them last year and they're a great bunch!</p><p>As a parting gift, here are two pieces of early concept art from our work at New Avalon. To the right is the brilliant character work from Yishu Ci (<a href="https://www.artstation.com/ciyishu">https://www.artstation.com/ciyishu</a>) and below is the amazing robots from Esther Wu (<a href="https://www.artbywu.com/">https://www.artbywu.com/</a>). I hope I get to work with them both again!</p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiszgivhmqt7lJOnVnEuEfbCaEq2LzWhOWVSMd-wrdhJ-y4edvBXgN-Rc05GiJYNJrlGbV_3kx13TnKPiT-AzK7trJihZWp34apj1d64G8VuYVsyUlI658UZWNf3C_3bhpVpXs99XrEURof72wJFT15HLLuQj49ghTgvlFMRQ1diWF4qNZ5MPC9vxuH/s2223/battle_logo_sm.jpg" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1250" data-original-width="2223" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiszgivhmqt7lJOnVnEuEfbCaEq2LzWhOWVSMd-wrdhJ-y4edvBXgN-Rc05GiJYNJrlGbV_3kx13TnKPiT-AzK7trJihZWp34apj1d64G8VuYVsyUlI658UZWNf3C_3bhpVpXs99XrEURof72wJFT15HLLuQj49ghTgvlFMRQ1diWF4qNZ5MPC9vxuH/w640-h360/battle_logo_sm.jpg" width="640" /></a></div><br /><br /><p><br /></p>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-1385093180449654952022-12-06T13:53:00.008-08:002022-12-06T13:53:51.091-08:00Some Podcast Appearances!<p>No news about New Avalon yet (we're still working on our prototype right now) but I was invited to speak on two podcasts recently to talk about anti-cheat. Links below!</p><p><a href="https://podcasts.apple.com/us/podcast/the-problem-with-kernel-mode-anti-cheat-software-ml-b-side/id1252417787?i=1000588988498">Malicious Life - The Problem With Kernel-Mode Anti-Cheat Software [ML B-Side]</a></p><p><a href="https://podcasts.apple.com/us/podcast/gaming-the-gamers/id1562732597?i=1000586163386">Cheat! - Gaming the Gamers</a><br /></p>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-71711338302036007742022-07-28T19:29:00.002-07:002022-07-28T19:29:25.505-07:00Going indie!<p>Another update!</p><p>For the last 10 months or so I've been doing consulting work for game studio startups, helping them with their tech. In particular I've helped with game networking, server performance & infrastructure, and security. At least one of the games is coming out soon and I'll make sure I post about it here.</p><p>While that's going on I've been exploring starting a game studio, after a few attempts trying with a Venture Capital compatible model I've changed course and instead started a small indie style studio. The studio is funded for now from my consulting work and hopefully augmented with additional funding from publishers or platforms in the future. The goal is to create a studio where creative people come together to build games to be proud of that find an audience sufficient to power the next project.</p><p>So here's the studio, New Avalon, our first project is a single player narratively driven strategy game. Not much to show yet, but I'm excited for the future!</p>
<blockquote class="twitter-tweet"><p dir="ltr" lang="en">Hey everyone, brand new tiny game studio here! We've just started working on the prototype for our first game.<br /><br />Soon we'll be looking for developers to join us to help us create the foundations of the game and to help us craft a pitch for publishers. <br /><br />Watch this space!</p>— New Avalon (@NewAvalonGames) <a href="https://twitter.com/NewAvalonGames/status/1550624426743308288?ref_src=twsrc%5Etfw">July 22, 2022</a></blockquote> <script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-36204036533577575922021-09-16T06:00:00.012-07:002022-03-24T17:39:53.835-07:00An Epic Games Portfolio<p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://cdn2.unrealengine.com/Unreal+Engine%2Feg-logo-filled-1255x1272-0eb9d144a0f981d1cbaaa1eb957de7a3207b31bb.png" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="800" data-original-width="789" height="200" src="https://cdn2.unrealengine.com/Unreal+Engine%2Feg-logo-filled-1255x1272-0eb9d144a0f981d1cbaaa1eb957de7a3207b31bb.png" width="197" /></a></div>My time at Epic Games has come to an end. It was brief but I learned a lot, I got to dust off some long unused skills (management) and got to work with some great people. Unfortunately the pandemic prevented me from spending time at Epic Games HQ or meeting many Epic employees in person but I'm sure the practice I got with remote work (and management) will come in handy!<br /><p></p><p>I joined Epic Games as a Principal Engineer on the Fortnite anti-cheat team but soon moved over to a Tech Director role to take over management responsibilities for the anti-cheat team and the related Gameplay Integrity team. This position had me looking after the anti-cheat technology for Epic Games (primarily for Fortnite) as well as being responsible for the competitive integrity of Fortnite in general (specifically Fortnite competitive events).</p><p>Beside management duties I also found some time for technical work but unlike previously almost all of my technical accomplishments were invisible to players. Over the course of the year I worked on technology for Fortnite all over the stack, including tech in the game client, game server, build systems, analytics pipeline, services infrastructure and operations tooling. This tech was usually anti-cheat or security related but occasionally I'd work on other areas such as performance or reliability issues.</p><p>The one exception to the behind the scenes nature of my work was the occasional feature or bug fixes for Unreal Engine. In particular in <a href="https://docs.unrealengine.com/4.27/en-US/WhatsNew/Builds/ReleaseNotes/4_27/">UE4.27</a> some of my work is mentioned in the release notes (and more of my changes can probably be found in the semi-public Github or Perforce logs). It was definitely a privilege to contribute (in a small way) to a game engine that I've spent so many years working with!</p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-qfGitx-vEpc/YS_pcXAkuHI/AAAAAAAA7GI/wzOQLobKE_s7JfphsQLAqkMzIS2wnInqACLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"></a><div class="separator" style="clear: both; text-align: center;"><a href="https://lh3.googleusercontent.com/-qfGitx-vEpc/YS_pcXAkuHI/AAAAAAAA7GI/wzOQLobKE_s7JfphsQLAqkMzIS2wnInqACLcBGAsYHQ/image.png" style="margin-left: 1em; margin-right: 1em;"></a><a href="https://lh3.googleusercontent.com/-5NbRV8d1-Sg/YS_p9i9yXrI/AAAAAAAA7GY/U_3hTToqaO0REnsHaCtWbjlGI-G_CBzSQCLcBGAsYHQ/image.png" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img data-original-height="128" data-original-width="750" height="109" src="https://lh3.googleusercontent.com/-5NbRV8d1-Sg/YS_p9i9yXrI/AAAAAAAA7GY/U_3hTToqaO0REnsHaCtWbjlGI-G_CBzSQCLcBGAsYHQ/w640-h109/image.png" width="640" /></a></div><table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody><tr><td style="text-align: center;"><a href="https://lh3.googleusercontent.com/-zNZqsDvDHqY/YS_qH684PqI/AAAAAAAA7Gc/IRbwTiIamRQyp9VdNZcNyJSHVmJ4Q1F4ACLcBGAsYHQ/image.png" style="margin-left: auto; margin-right: auto;"><img data-original-height="109" data-original-width="749" height="93" src="https://lh3.googleusercontent.com/-zNZqsDvDHqY/YS_qH684PqI/AAAAAAAA7Gc/IRbwTiIamRQyp9VdNZcNyJSHVmJ4Q1F4ACLcBGAsYHQ/w640-h93/image.png" width="640" /></a></td></tr><tr><td class="tr-caption" style="text-align: center;">Two examples of my engine changes from the release notes<br /></td></tr></tbody></table><br /><br /><br /></div><br /><p></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p><br /></p><p>Next up after Epic I plan to be more of a generalist and spend less of my time on anti-cheat issues. The form it'll take is still up in the air but when I have something to announce I'll try and remember to post it here!</p>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0Cary, NC, USA35.79154 -78.78111697.4813061638211522 -113.9373669 64.101773836178836 -43.6248669tag:blogger.com,1999:blog-1404660204966454042.post-16509319755732364782020-09-02T09:18:00.015-07:002022-05-27T02:21:23.469-07:00A VALORANT Portfolio<div class="separator"><p style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-ng8TwvIjbrw/X0_HL8JOAMI/AAAAAAAA5KI/7t4qZzygtvwxQpajqy17zM8fE8zzgJNcACLcBGAsYHQ/s396/vanguard%2Bspray.png" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" data-original-height="396" data-original-width="305" height="104" src="https://1.bp.blogspot.com/-ng8TwvIjbrw/X0_HL8JOAMI/AAAAAAAA5KI/7t4qZzygtvwxQpajqy17zM8fE8zzgJNcACLcBGAsYHQ/w80-h104/vanguard%2Bspray.png" width="80" /></a></div>I'm changing jobs (after almost 7 years at Riot, that's a long run!) and I wanted to post a bunch of links to public facing stuff from my time at Riot, mostly VALORANT stuff from the past year, this is very self indulgent but think of it as a portfolio of sorts.<p></p></div><h2 style="text-align: left;">Writing</h2><p><a href="https://technology.riotgames.com/news/demolishing-wallhacks-valorants-fog-war" id="https://technology.riotgames.com/news/demolishing-wallhacks-valorants-fog-war" name="https://technology.riotgames.com/news/demolishing-wallhacks-valorants-fog-war">Riot Tech Blog - Fog of War<br /></a></p><p><a href="https://playvalorant.com/en-us/news/game-updates/01-on-raze-and-anti-cheat/" id="https://playvalorant.com/en-us/news/game-updates/01-on-raze-and-anti-cheat/" name="https://playvalorant.com/en-us/news/game-updates/01-on-raze-and-anti-cheat/">Game Update 01: ON RAZE AND ANTI-CHEAT</a></p><p><a href="https://playvalorant.com/en-gb/news/game-updates/02-on-vanguard-errors-solutions-gun-skins-and-agent-progression/" id="https://playvalorant.com/en-gb/news/game-updates/02-on-vanguard-errors-solutions-gun-skins-and-agent-progression/" name="https://playvalorant.com/en-gb/news/game-updates/02-on-vanguard-errors-solutions-gun-skins-and-agent-progression/">Game Update 02: ON VANGUARD ERRORS/SOLUTIONS, GUN SKINS, AND AGENT PROGRESSION</a></p><p><a href="https://playvalorant.com/en-us/news/game-updates/04-on-peeker-s-advantage-ranked/" id="https://playvalorant.com/en-us/news/game-updates/04-on-peeker-s-advantage-ranked/" name="https://playvalorant.com/en-us/news/game-updates/04-on-peeker-s-advantage-ranked/">Game Update 04: ON PEEKER’S ADVANTAGE & RANKED</a></p><p><a href="https://playvalorant.com/en-gb/news/game-updates/ask-valorant-5/" id="https://playvalorant.com/en-gb/news/game-updates/ask-valorant-5/" name="https://playvalorant.com/en-gb/news/game-updates/ask-valorant-5/">Ask Valorant #5</a></p><p><a href="https://playvalorant.com/en-us/news/dev/valorant-anti-cheat-what-why-and-how/" id="https://playvalorant.com/en-us/news/dev/valorant-anti-cheat-what-why-and-how/" name="https://playvalorant.com/en-us/news/dev/valorant-anti-cheat-what-why-and-how/">Dev Blog: Anti-cheat What, Why, and How</a></p><p><a href="https://playvalorant.com/en-us/news/dev/valorant-anti-cheat-cheater-reported/">Dev Blog: VALORANT ANTI-CHEAT: CHEATER, REPORTED!</a></p><p></p><p><a href="https://old.reddit.com/r/VALORANT/comments/fxyzbe/cheater_banned/fmxnpll/">Reddit comment about the first cheaters banned</a></p><p><a href="https://old.reddit.com/r/VALORANT/comments/fzxdl7/anticheat_starts_upon_computer_boot/fn6yqbe/">Reddit comment about running a driver at boot</a></p><p><a href="https://old.reddit.com/r/VALORANT/comments/g9aoap/upcoming_vanguard_changes/fosbtkm/" id="https://old.reddit.com/r/VALORANT/comments/g9aoap/upcoming_vanguard_changes/fosbtkm/" name="https://old.reddit.com/r/VALORANT/comments/g9aoap/upcoming_vanguard_changes/fosbtkm/">Reddit comment about a performance regression</a></p><p><a href="https://old.reddit.com/user/riotarkem" id="https://old.reddit.com/user/riotarkem" name="https://old.reddit.com/user/riotarkem">My Riot Reddit Account history</a></p><p>Tweet about how many cheaters exist in games and how that feels for players:</p><blockquote class="twitter-tweet"><p dir="ltr" lang="en">There wasn't a right or wrong answer to this poll but I do have some background to give. <br /><br />It's hard to measure cheating rates because we often have imperfect detection. Devs normally estimate cheating populations by extrapolating from detections, player reports and research. <a href="https://t.co/UOzBTkVkNy">https://t.co/UOzBTkVkNy</a></p>— Paul Chamberlain (@arkem) <a href="https://twitter.com/arkem/status/1276308820575186944?ref_src=twsrc%5Etfw">June 26, 2020</a></blockquote> <script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script><p><br /></p><h2>Interviews / News clips</h2><div><br /></div><div>I did a lot of interviews and got asked for quotes many times, I didn't do a good job keeping track of how those quotes and interviews were used but here are some of them that I could find after the fact.</div><p><a href="https://www.polygon.com/2020/3/2/21158401/valorant-how-riot-games-made-something-new">Polygon: Valorant: How Riot finally made something new</a></p><div><a href="https://www.youtube.com/watch?v=Y9_uEx7Iwa8">Nerd Street Gamers Interview Video</a></div><p><a href="https://www.invenglobal.com/articles/10547/valorant-interview-with-the-developers-part-2-valorant-has-prepared-to-deal-with-hacks-from-the-very-beginning">InvenGlobal: Valorant has prepared to deal with hacks from the very beginning</a></p><p><a href="https://www.polygon.com/2020/5/12/21256388/valorant-riot-games-anti-cheat-vanguard-closed-beta-bans">Polygon: Valorant team bans over 8,000 cheaters in closed beta</a></p><div><p><a href="https://www.ign.com/articles/valorant-cheaters-remain-banned-after-beta">IGN: Valorant Cheaters Remain Banned After Beta</a></p></div><div><a href="https://www.chinajoy.net/cgdcen/yjjb/keynote/paul-chamberlain/">ChinaJoy Keynote official page</a></div><p><a href="https://www.youtube.com/watch?v=gOdnvU4rYO4">ChinaJoy Keynote Video</a></p><p><a href="https://www.vice.com/en_us/article/bv857z/gamerdoc-catching-banning-cheaters-hackers-overwatch-valorant">VICE: The Vigilante Hunting Down Cheaters in Video Games</a></p><p><a href="https://www.riotgames.com/en/news/a-message-about-vanguard-from-our-security-privacy-teams">Riot: A Message About Vanguard From Our Security & Privacy Teams</a></p><p><a href="https://www.youtube.com/watch?v=4W_1zA6j9FU">Engadget: Valorant’s always-on anti-cheat system, Vanguard, is invasive AF</a></p><h2 style="text-align: left;"><br /></h2><h2 style="text-align: left;">Valorant Game Features</h2><p>Most of what I build for Valorant was behind the scenes technical systems there are some more visible features that I want to highlight below.</p><p><b>Fog of War</b></p><blockquote class="twitter-tweet"><p dir="ltr" lang="en">Here's the "with Fog of War enabled" clip from the article <a href="https://t.co/FdTY2tWICz">https://t.co/FdTY2tWICz</a> <a href="https://t.co/TK6bdBsm1F">pic.twitter.com/TK6bdBsm1F</a></p>— Paul Chamberlain (@arkem) <a href="https://twitter.com/arkem/status/1250117284678385664?ref_src=twsrc%5Etfw">April 14, 2020</a></blockquote> <script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script><p><b>Map targeted abilities</b></p><p><img alt="omen ultimate minimap" height="354" src="https://mobalytics.gg/wp-content/uploads/2020/04/omen-ultimate-minimap.jpg" width="629" /></p><p><b>Minimap footstep audio circles</b></p><p><img alt="Valorant: Beginners guide – Gameplay mechanics you need to know" src="https://win.gg/ezoimgfmt/cdn-images.win.gg/wp/uploads/2022/01/Valorant-footstep-audio-range.jpg?ezimgfmt=ng:webp/ngcb13" /></p><p><b>Player facing no-hud/cinematic mode</b></p><p></p><div class="separator" style="clear: both; text-align: center;"><a href="https://1.bp.blogspot.com/-gDv7Zn4m7JE/X0_Fw-SKk0I/AAAAAAAA5J8/88z8J1Uk0qkpkGw2tsA0b1QUzZJ0g0n2ACLcBGAsYHQ/s2048/valorant%2Bnohud%2Bscreenshot.png" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1150" data-original-width="2048" src="https://1.bp.blogspot.com/-gDv7Zn4m7JE/X0_Fw-SKk0I/AAAAAAAA5J8/88z8J1Uk0qkpkGw2tsA0b1QUzZJ0g0n2ACLcBGAsYHQ/s640/valorant%2Bnohud%2Bscreenshot.png" width="640" /></a></div><br /><b><br /></b><p></p><p><b>Early grenade physics (~2015 version)</b></p><div class="separator" style="clear: both; text-align: center;"><iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.blogger.com/video.g?token=AD6v5dwbm2Tf_soz9Kbc7_b7Qa9CZsbxndp0O6_HiloNqIr3sLwPfqvkk1YwxXDK7SMAaRSp3pgJ4CCJ-ECdkcZd4w' class='b-hbp-video b-uploaded' frameborder='0'></iframe></div><p><br /></p><p><b>Character select (the first version ~2015)</b></p><p><img alt="VALORANT Early Thoughts | Phantom Distillery" height="378" src="https://phantomdistillery.com/VALORANT-Early-Analysis/characterselect.png" width="670" /></p><p><b>Security error messages / ban messages (UI and plumbing)</b></p><p><a href="https://twitter.com/arkem/status/1193343334636326912">https://twitter.com/arkem/status/1193343334636326912</a></p><blockquote class="twitter-tweet"><p dir="ltr" lang="en">Here's me being banned from Project A. I am better at security than at art but my "HACKER DETECTED" screen is a thing of beauty. <a href="https://t.co/3N0953yqq0">pic.twitter.com/3N0953yqq0</a></p>— Paul Chamberlain (@arkem) <a href="https://twitter.com/arkem/status/1193343334636326912?ref_src=twsrc%5Etfw">November 10, 2019</a></blockquote> <script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script><p><a href="https://twitter.com/Ninja/status/1278388868534894599">https://twitter.com/Ninja/status/1278388868534894599</a></p><blockquote class="twitter-tweet"><p dir="ltr" lang="sv">Feelsgoodman <a href="https://t.co/xxhuYomrVF">pic.twitter.com/xxhuYomrVF</a></p>— Ninja (@Ninja) <a href="https://twitter.com/Ninja/status/1278388868534894599?ref_src=twsrc%5Etfw">July 1, 2020</a></blockquote> <script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script><p><br /></p><p><b>Lots of behind the scenes stuff</b> </p><p>Anti-cheat and security systems, network encryption, performance optimization, UI tweaks, gameplay telemetry system, engine upgrades, middleware integrations, code analysis tools and lots more that I've already forgotten.</p><p><br /></p>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-13344605786965665222020-06-02T07:14:00.001-07:002020-06-02T07:14:25.904-07:00VALORANT released!<div class="separator" style="clear: both; text-align: left;">
It's been many years in the making but VALORANT has been released!</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
More information here <a href="https://playvalorant.com/" target="_blank">https://playvalorant.com</a> or watch this animated trailer.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<iframe width="320" height="266" class="YOUTUBE-iframe-video" data-thumbnail-src="https://i.ytimg.com/vi/e_E9W2vsRbQ/0.jpg" src="https://www.youtube.com/embed/e_E9W2vsRbQ?feature=player_embedded" frameborder="0" allowfullscreen></iframe></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div>
<br /></div>
Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-45471322724393505012020-04-14T10:12:00.000-07:002020-04-15T09:42:06.433-07:00VALORANT is here!The <a href="https://playvalorant.com/" target="_blank">VALORANT</a> closed beta is open in NA and EU and the response has been overwhelmingly positive!<br />
<br />
However, the interest from cheaters has also been very high and we had to ban our first cheaters during the first week of the closed beta.<br />
<br />
<blockquote class="twitter-tweet">
<div dir="ltr" lang="en">
Well it sucks, but today we had to ban our first cheater (and it looks like more bans are on the horizon).<br />
<br />
I was hoping for a little more time before this fight kicked off but we're in it now and we're ready.</div>
— Paul Chamberlain (@arkem) <a href="https://twitter.com/arkem/status/1248331794446860288?ref_src=twsrc%5Etfw">April 9, 2020</a></blockquote>
<br />
There's also been a lot of interest around the details of the Riot Vanguard anti-cheat system, especially the kernel component. I ended up <a href="https://old.reddit.com/r/VALORANT/comments/fzxdl7/anticheat_starts_upon_computer_boot/fn6yqbe/" target="_blank">talking on reddit about it</a> a fair bit and even got some interest from the gaming press (I sent written statements to some outlets, <strike>don't know if they'll run the articles though</strike> Edit: <a href="https://arstechnica.com/gaming/2020/04/ring-0-of-fire-does-riot-games-new-anti-cheat-measure-go-too-far/" target="_blank">Ars Technica</a>, <a href="https://kotaku.com/riot-addresses-concerns-over-valorants-always-on-anti-c-1842848668" target="_blank">Kotaku</a> and <a href="https://www.polygon.com/2020/4/15/21221046/valorant-anti-cheat-beta-system-launch-boot-up-drivers-uninstall" target="_blank">Polygon</a>).<br />
<br />
I also wrote a big article about the <a href="https://technology.riotgames.com/news/demolishing-wallhacks-valorants-fog-war" target="_blank">VALORANT Fog of War system</a> that I'm really proud of. It has some gameplay clips and some illustrations I made as well as a good overview of the road we took to shipping that system.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://1.bp.blogspot.com/-7vZjNSVpZDA/XpXunaOYCEI/AAAAAAAA4r8/IdIwXKcqLbst828s7uxjpDnF1Kouc4ROACLcBGAsYHQ/s1600/valanticheat_10.gif" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="450" data-original-width="800" height="360" src="https://1.bp.blogspot.com/-7vZjNSVpZDA/XpXunaOYCEI/AAAAAAAA4r8/IdIwXKcqLbst828s7uxjpDnF1Kouc4ROACLcBGAsYHQ/s640/valanticheat_10.gif" width="640" /></a></div>
<br />
It's been a really busy first week but it's all looking really promising!<br />
<script async="" charset="utf-8" src="https://platform.twitter.com/widgets.js"></script>
Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-49018255332436807292019-11-13T23:50:00.000-08:002019-11-13T23:51:16.609-08:00What have I really been doing these last few years?In addition to everything mentioned in the last post, I've been busy making a character based tactical shooter code-named "Project A". I'm the lead for anti-cheat and security but I've also put a lot of effort into many other parts of the game.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><br />
<a href="https://1.bp.blogspot.com/-L7Rml2dsmNY/Xc0F3-FA51I/AAAAAAAA3rE/W4Go7n-_5m42UJuuq0KTnjG1T1sJs26gwCLcBGAsYHQ/s1600/final_5dc76972bae7b50014b52024_668355.gif" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="405" data-original-width="720" src="https://1.bp.blogspot.com/-L7Rml2dsmNY/Xc0F3-FA51I/AAAAAAAA3rE/W4Go7n-_5m42UJuuq0KTnjG1T1sJs26gwCLcBGAsYHQ/s1600/final_5dc76972bae7b50014b52024_668355.gif" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Here's me getting banned by my own anti-cheat system.</td></tr>
</tbody></table>
Chances are the next few blog posts here will be about my work on Project A as the game gets closer to launch, there's not much I can share right now but more details are coming in 2020.<br />
<div>
<br /></div>
<div>
If you're interested in more information about Project A, here's our first announce video: <a href="https://www.youtube.com/watch?v=4iGU6PctOBg" target="_blank">Project A: Riot’s Tactical FPS Announcement</a></div>
<div>
<br /></div>
Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-31051892811663148452018-11-23T23:00:00.001-08:002018-11-23T23:00:56.058-08:00What have I been doing these last few years?I've been relatively quiet on this blog since I started working at Riot Games in 2013 in part because my day job has been more on the game development side than the security side so there haven't been a lot of security topics worth writing about. I thought since it has been five years now I'd do a round up now that I have a few years of bits and pieces that have some security crossover!<br />
<br />
When I started at Riot I worked on the infosec team on a variety of things (primarily incident response and bug bounty) here's some of the cool work that team did:<br />
<br />
<a href="https://engineering.riotgames.com/news/running-bug-bounty-program">Running a Bug Bounty Program</a> - Blog post about Riot's approach to bug bounties<br />
<a href="https://engineering.riotgames.com/news/evolution-security-riot">The Evolution of Security at Riot</a> - Overview of Riot's infosec program<br />
<a href="https://github.com/RiotGames/cloud-inquisitor">Cloud Inquisitor</a> - Security monitoring and policy enforcement tool for AWS (open source)<br />
<br />
After I left Riot infosec I moved onto League of Legends where I worked on cheat detection and prevention systems as well as some networking and metrics collection changes. I wrote code that was deployed to hundreds of millions of computers and eventually led to some of things described in these articles:<br />
<br />
<a href="https://nexus.leagueoflegends.com/en-us/2018/10/dev-removing-cheaters-from-lol/">Removing Cheaters From LoL</a> - Player facing overview of Riot's anti-cheat activities<br />
<a href="https://engineering.riotgames.com/news/riots-approach-anti-cheat">Riot's Approach to Anti-cheat</a> - A good overview of Riot's anti-cheat strategies and tech<br />
<a href="https://www.gamesindustry.biz/articles/2017-03-06-riot-games-wins-USD10-million-in-leaguesharp-suit">Riot Games wins $10 million in LeagueSharp suit</a> - A tech and legal battle I influenced<br />
<a href="https://dotesports.com/league-of-legends/news/riot-anticheat-riotofpenguins-14841">Riot’s anti-cheat team just took down a huge scripting provider</a> - Another battle for the team<br />
<br />
Eventually I left League of Legends and its anti-cheat team behind. I entrusted it to the extra-ordinarily talented Nemi and Michael who by day build great anti-cheat systems and by night run the blog <a href="https://www.triplefault.io/">https://www.triplefault.io/</a>.<br />
<br />
In particular check out these great posts from them:<br />
<a href="https://www.triplefault.io/2018/05/spurious-db-exceptions-with-pop-ss.html">Spurious #DB exceptions with the "MOV SS" and "POP SS" instructions (CVE-2018-8897)</a><br />
<a href="https://www.triplefault.io/2017/09/enumerating-process-thread-and-image.html">Enumerating process, thread, and image load notification callback routines in Windows</a><br />
<a href="https://www.triplefault.io/2017/08/detecting-debuggers-by-abusing-bad.html">Detecting debuggers by abusing a bad assumption within Windows</a><br />
<br />
So if I haven't been working on League of Legends what have I been doing? I've been exploring game development (and a helping of security/anti-cheat work) for on a new game at Riot, it's a project that I'm super excited about but one that's not ready for the limelight. There's a good chance my next blog post here will be pointing you all at the project so stay tuned!Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-69318177380677081502015-09-27T21:59:00.002-07:002015-09-27T22:54:36.194-07:00KPROCESS - InstrumentationCallback - Get callbacks on return from kernel modeLong time no see everybody!<br />
<br />
I was pointed at a really interesting article this week and thought I'd share.<br />
<br />
It turns out in Windows Vista x64 and later you can ask the kernel to invoke a callback on transition out of kernel mode. KPROCESS has a field called InstrumentationCallback that you can set with a call to NtSetInformationProcess with a ProcessInfoClass of 0x28 and a InputBuffer that contains the pointer to your callback.<br />
<br />
Advantages:<br />
<br />
<ul>
<li>User mode only, no driver or kernel debugger required</li>
<li>Affects the entire process (including injected threads)</li>
</ul>
<div>
<br /></div>
Disadvantages:<br />
<br />
<ul>
<li>Windows x64 only</li>
<li>Apparently doesn't work on WOW64 (though maybe some jiggery-pokery could get it done)</li>
<li>Required Dr7 to be set in most cases so not great at catching malicious actors</li>
</ul>
<br />
The original article: <a href="http://www.codeproject.com/Articles/543542/Windows-x-system-service-hooks-and-advanced-debu">Windows x64 system service hooks and advanced debugging</a> and check out the author's blog <a href="http://everdox.blogspot.com/">http://everdox.blogspot.com/</a> for other interesting RE posts.Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-33133503045809839242013-11-05T21:15:00.000-08:002013-11-05T21:38:47.080-08:00Authenticode and Antivirus Detection RevisitedIt's time to revisit code signing and antivirus detection! Two years ago I looked into whether or not Authenticode signatures (Microsoft object code signing for PE files) influenced the decisions of antivirus engines.<br />
<br />
In the <a href="http://memeover.arkem.org/2011/08/authenticode-and-antivirus-detection.html">first part</a> I described the process of finding, signing and testing some malware with <a href="http://www.virustotal.com/">VirusTotal</a> and it appeared that adding an Authenticode signature to a known piece of malware drastically lowered its detection rate. After an astute observation in the comments and some thinking about it, I decided this was more likely due to the fragility of the signatures created by antivirus vendors. In the <a href="http://memeover.arkem.org/2011/08/authenticode-and-antivirus-detection_08.html">second part</a> I test this theory by using the VirusTotal API to test 100 malware samples to compare the response of adding an Authenticode signature to changing the binary in other ways. The result largely confirmed that rather than code signing that was defeating the antivirus scans, it was that the binary was changing at all.<br />
<br />
These results were ultimately unsatisfying, partly because of the surprising fragility of the antivirus signatures but also because of the test methodology. VirusTotal is a wonderful resource but is only testing the core antivirus engine of these security products which these days is only a small part of the protective services provided by security software. Additionally, the certificate that the malware was signed with was not endorsed by any certificate authority and was not deemed trusted by the operating system.<br />
<br />
This time around I investigated how the actual security products responded, and used a trusted certificated to sign the malware. Unlike last time, the number of engines tested is much lower and the number of samples used is only one. Turns out that this kind of testing is very time intensive if you don't already have the infrastructure set up. While I did have some free time during the US Government shutdown while waiting for a US visa, there's a limit to how many virtual machines I felt like setting up.<br />
<br />
<b>Part 1 - The Malware</b><br />
This time the malware sample used is VPN-Pro.exe (MD5: 8eda7dfa4ec4ac975bb12d2a3186bbeb) as contributed by the redoubtable <a href="https://twitter.com/headhntr">@headhntr</a> to the <a href="http://syrianmalware.com/">Syrian Malware Samples Project</a>. As described by the <a href="https://citizenlab.org/2013/06/a-call-to-harm/">Citizenlab analysis</a> it is a trojanized version of Freegate 7.35 written using .NET 3.5 that drops the ShadowTech RAT. The campaign was targeted at dissidents in Syria, make sure that you check out the analysis, it's fascinating but a little outside the scope of this post.<br />
<br />
When Citizenlab submitted VPN-Pro.exe to VirusTotal in June 2013 it was detected by 5/46 antivirus engines. When I checked in October the <a href="https://www.virustotal.com/en/file/829e137279f691e493c211108b62c8e15b079bd619ba19ad388450878e0585d0/analysis/">VirusTotal report</a> had been updated to show detection by 34/47 antivirus engines.<br />
<br />
<b>Part 2 - The Antivirus Suites</b><br />
The test environment is a series of Windows 8.1 Virtual Machines with the following security software suites installed.<br />
<br />
<ul>
<li>Sophos Endpoint Client Protection</li>
<li>McAfee All Access</li>
<li>Norton 360</li>
<li>Windows Defender (as installed by default with Windows 8.1)</li>
</ul>
<div>
In addition Chrome was installed in each VM, and each sample was submitted to VirusTotal.</div>
<div>
<br /></div>
<div>
<b>Part 3 - The Transformations</b></div>
<div>
Six versions of the sample were tested, the original and five transformed versions.</div>
<div>
<br /></div>
<div>
<u>1. Padded</u></div>
<div>
VPN-Pro.exe with 1024 'A' characters appended to the end.</div>
<div>
SHA256: <a href="https://www.virustotal.com/en/file/54d9f5767ec3a7aba6754dacc998d57bb54e793750f2e5b1e63e37cc9c43da6e/analysis/1382678003/">54d9f5767ec3a7aba6754dacc998d57bb54e793750f2e5b1e63e37cc9c43da6e</a></div>
<div>
<br /></div>
<div>
<u>2. Random Padding</u></div>
<div>
VPN-Pro.exe with random bytes added to match the length of the signed version.</div>
<div>
SHA256: <a href="https://www.virustotal.com/en/file/8428aa9dfa69438d98b0008b0dc7c9e8135889d893a77d5536aacf8b7e1ad6e7/analysis/1382679731/">8428aa9dfa69438d98b0008b0dc7c9e8135889d893a77d5536aacf8b7e1ad6e7</a></div>
<div>
<br /></div>
<div>
<u>3. Authenticode (Self-signed certificate)</u></div>
<div>
VPN-Pro.exe signed with a test certificate (as describe in part 1 of this series)</div>
<div>
SHA256: <a href="https://www.virustotal.com/en/file/f8efd5ad3ad13e218b71dece73766c87e57c80d5f7a1fd78f319312baa11bcf7/analysis/1382678288/">f8efd5ad3ad13e218b71dece73766c87e57c80d5f7a1fd78f319312baa11bcf7</a></div>
<div>
<br /></div>
<div>
<u>4. Damaged Authenticode (Self-signed certificate)</u></div>
<div>
VPN-Pro.exe as prepared in number 3 but with roughly 10% of the signature bytes replaced randomly.</div>
<div>
SHA256: <a href="https://www.virustotal.com/en/file/1ddf2de1bb8d289b6b77843bc2b9a685d31d0f17371691e3f4b81faa383c7769/analysis/1382678303/">1ddf2de1bb8d289b6b77843bc2b9a685d31d0f17371691e3f4b81faa383c7769 </a></div>
<div>
<br /></div>
<div>
<u>5. Authenticode (Trusted certificate)</u></div>
<div>
VPN-Pro.exe signed with a trusted code signing certificate from <a href="http://startssl.com/">StartSSL.com</a> (thanks to the anonymous benefactor that helped me with this part).</div>
<div>
SHA256: <a href="https://www.virustotal.com/en/file/d9deeaa7762072d5cb8f99ecea7c1acf32354ba4486f9afb01f4404149b919fd/analysis/1383538841/">f8efd5ad3ad13e218b71dece73766c87e57c80d5f7a1fd78f319312baa11bcf7</a></div>
<div>
<br /></div>
<div>
<b>Part 4 - The Method</b></div>
<div>
Each VM had the trial version (as found on the vendor's website) of the security software installed. Windows and the security software were then updated. Next Chrome was used to download each sample and the reactions of the browser and security software were recorded. If there was no reaction from the security software or browser a manual scan was initiated where possible. In the case of Windows Defender the same tests were undertaken using Internet Explorer as well as Chrome.</div>
<div>
<br /></div>
<div>
<b>Part 5 - The Results</b><br />
<u><br /></u>
<u>Summary</u></div>
<div>
<table cellpadding="0" cellspacing="0" dir="ltr" style="font-family: arial,sans,sans-serif; font-size: 13px; table-layout: fixed;"><colgroup><col width="120"></col><col width="78"></col><col width="78"></col><col width="77"></col><col width="77"></col><col width="75"></col><col width="75"></col></colgroup><tbody>
<tr style="height: 17px;"><td style="border-bottom: 1px solid #ccc; border-left: 1px solid #ccc; border-right: 1px solid #ccc; border-top: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Original</td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Padded</td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Random Padding</td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Damaged Signature</td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Self Signed</td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; border-top-color: rgb(204, 204, 204); border-top-style: solid; border-top-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Trusted Signature</td></tr>
<tr style="height: 17px;"><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; border-left-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">McAfee All Access</td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td></tr>
<tr style="height: 17px;"><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; border-left-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Norton 360</td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td></tr>
<tr style="height: 17px;"><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; border-left-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Sophos Endpoint</td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: lime; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td></tr>
<tr style="height: 17px;"><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; border-left-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Windows Defender</td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: red; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: yellow; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: yellow; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: yellow; border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="background-color: lime; border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;"><b><span style="color: magenta;">Mild Warn</span></b></td></tr>
<tr style="height: 17px;"><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; border-left-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Virus Total</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; direction: ltr; padding: 0 3px; text-align: center; vertical-align: bottom;">34/47</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; direction: ltr; padding: 0 3px; text-align: center; vertical-align: bottom;">7/47</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; direction: ltr; padding: 0 3px; text-align: center; vertical-align: bottom;">7/46</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; direction: ltr; padding: 0 3px; text-align: center; vertical-align: bottom;">7/47</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; direction: ltr; padding: 0 3px; text-align: center; vertical-align: bottom;">7/47</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; direction: ltr; padding: 0 3px; text-align: center; vertical-align: bottom;">8/47</td></tr>
<tr style="height: 17px;"><td style="border-bottom: 1px solid #ccc; border-left: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;"></td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;"></td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;"></td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;"></td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;"></td><td style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;"></td></tr>
<tr style="height: 17px;"><td style="background-color: lime; border-bottom: 1px solid #ccc; border-left: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td colspan="2" rowspan="1" style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Nothing detected</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td></tr>
<tr style="height: 17px;"><td style="background-color: yellow; border-bottom: 1px solid #ccc; border-left: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td colspan="3" rowspan="1" style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Warning, this may harm your PC</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td></tr>
<tr style="height: 17px;"><td style="background-color: #ff9900; border-bottom: 1px solid #ccc; border-left: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td colspan="3" rowspan="1" style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Reputation based detection</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td></tr>
<tr style="height: 17px;"><td style="background-color: red; border-bottom: 1px solid #ccc; border-left: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td colspan="3" rowspan="1" style="border-bottom-color: rgb(204, 204, 204); border-bottom-style: solid; border-bottom-width: 1px; border-right-color: rgb(204, 204, 204); border-right-style: solid; border-right-width: 1px; direction: ltr; padding: 0px 3px; vertical-align: bottom;">Explicitly marked as virus</td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td><td style="border-bottom: 1px solid #ccc; border-right: 1px solid #ccc; padding: 0 3px; vertical-align: bottom;"></td></tr>
</tbody></table>
</div>
<div>
<u><br /></u>
<u>McAfee All Access</u></div>
<div>
<u><br /></u></div>
<div>
The original sample was detected as malware and automatically removed.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-uccKFviTaYY/Unm_YcS-NaI/AAAAAAAAfx4/Qn2Gu7JVJHA/s1600/mcafee-vpn-pro.exe.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="271" src="http://3.bp.blogspot.com/-uccKFviTaYY/Unm_YcS-NaI/AAAAAAAAfx4/Qn2Gu7JVJHA/s400/mcafee-vpn-pro.exe.png" width="400" /></a></div>
<div>
<br /></div>
<div>
All other samples were detected as malware and quarantined.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-zJkmQ7MB5Fs/Unm_4E56hsI/AAAAAAAAfyA/D3EOhrFml_w/s1600/mcafee-vpn-pro.signedrm.exe.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="300" src="http://1.bp.blogspot.com/-zJkmQ7MB5Fs/Unm_4E56hsI/AAAAAAAAfyA/D3EOhrFml_w/s400/mcafee-vpn-pro.signedrm.exe.png" width="400" /></a></div>
<div>
<br /></div>
<div>
Considering that on the VirusTotal scan McAfee did not detect any of the transformed samples, my guess is that the Quarantined dialog is shown on heuristic or binary reputation based matches while the automated removal dialog is for signature based matches.</div>
<div>
<br /></div>
<div>
<u>Norton 360</u></div>
<div>
<br /></div>
<div>
The original sample was detected as malware and automatically removed.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-1KkS8SlpNKY/UnnAkear3QI/AAAAAAAAfyI/rumPatwrz4s/s1600/norton-vpn-pro-1.exe.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="459" src="http://1.bp.blogspot.com/-1KkS8SlpNKY/UnnAkear3QI/AAAAAAAAfyI/rumPatwrz4s/s640/norton-vpn-pro-1.exe.png" width="640" /></a></div>
<div>
<br /></div>
<div>
The transformed samples were also all detected and automatically removed.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-LFElhI6CM0Q/UnnA0dB3MxI/AAAAAAAAfyQ/CYebGU_iyyM/s1600/norton-vpn-pro.signedrm-1.exe.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="459" src="http://2.bp.blogspot.com/-LFElhI6CM0Q/UnnA0dB3MxI/AAAAAAAAfyQ/CYebGU_iyyM/s640/norton-vpn-pro.signedrm-1.exe.png" width="640" /></a></div>
<div>
The difference being that all the transformed files were detected as the threat "WS.Reputation.1" and the Threat Type of "Insight Network Threat". This suggests to me that the cloud binary reputation service is flagging these files as harmful largely because they have note been seen before. Again, the Symantec engine did not detect the transformed files during the VirusTotal submission (I assume that Norton 360 uses the Symantec antivirus engine).</div>
<div>
<br /></div>
<div>
<u>Sophos Client Endpoint Protection</u></div>
<div>
<u><br /></u></div>
<div>
Again, the original sample was detected and removed.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-IVHOvxUAmcE/UnnCbq7YwpI/AAAAAAAAfyg/xy-GBuJ562s/s1600/sophos-chrome-vpn-pro.exe.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="512" src="http://2.bp.blogspot.com/-IVHOvxUAmcE/UnnCbq7YwpI/AAAAAAAAfyg/xy-GBuJ562s/s640/sophos-chrome-vpn-pro.exe.png" width="640" /></a></div>
<div>
<br /></div>
<div>
In fact all the transformations (except one) of VPN-Pro.exe were detected in the same way as the original and were tagged as Mal/Generic-S. The original was flagged as Mal/Generic-S on VirusTotal as well but the transformations weren't likewise flagged at the time, unsure whether this is due to some fuzzy matching or updated signatures.</div>
<div>
<br /></div>
<div>
However, the Authenticode version with the trusted certificate was downloaded without complaint. Considering that the self signed version was flagged as malicious, I'm drawn to conclude that the validity of the signature was taken into account.</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-FC1LaC6sm8s/UnnDjhTwVFI/AAAAAAAAfyw/kovFdtxASkY/s1600/sophos-vpn-pro.signedrm.exe.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="360" src="http://4.bp.blogspot.com/-FC1LaC6sm8s/UnnDjhTwVFI/AAAAAAAAfyw/kovFdtxASkY/s640/sophos-vpn-pro.signedrm.exe.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">A manual scan was run after the initial download completed.</td></tr>
</tbody></table>
<div>
<u><br /></u></div>
<div>
<u>Windows Defender</u></div>
<div>
<u><br /></u></div>
<div>
Windows Defender on Internet Explorer gave the largest variety of messages, here's all six:</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-o4WeD6g3IuU/UnnFPSCSVnI/AAAAAAAAfzM/cfrsM2h0RHA/s1600/ie-vpn-pro.exe.cropped.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="32" src="http://1.bp.blogspot.com/-o4WeD6g3IuU/UnnFPSCSVnI/AAAAAAAAfzM/cfrsM2h0RHA/s640/ie-vpn-pro.exe.cropped.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-DfpULnimYqg/UnnFOpqm3VI/AAAAAAAAfy8/8Re_N5Ymnv0/s1600/ie-vpn-pro.padded.exe.cropped.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="41" src="http://2.bp.blogspot.com/-DfpULnimYqg/UnnFOpqm3VI/AAAAAAAAfy8/8Re_N5Ymnv0/s640/ie-vpn-pro.padded.exe.cropped.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://1.bp.blogspot.com/-YYE5hapMNNw/UnnFOXuzCLI/AAAAAAAAfy4/NKr3Q8wG6HQ/s1600/ie-vpn-pro.randpad.exe.cropped.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="41" src="http://1.bp.blogspot.com/-YYE5hapMNNw/UnnFOXuzCLI/AAAAAAAAfy4/NKr3Q8wG6HQ/s640/ie-vpn-pro.randpad.exe.cropped.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://2.bp.blogspot.com/-TiX0eV1o6yk/UnnFQTr41VI/AAAAAAAAfzY/-6qX8O1VUdc/s1600/ie-vpn-pro.signed_damaged.exe.cropped.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="41" src="http://2.bp.blogspot.com/-TiX0eV1o6yk/UnnFQTr41VI/AAAAAAAAfzY/-6qX8O1VUdc/s640/ie-vpn-pro.signed_damaged.exe.cropped.png" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-vOAEraOR3qk/UnnFPQFvWMI/AAAAAAAAfzE/G2oGkEvqD_A/s1600/ie-vpn-pro.signed.exe.cropped.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="65" src="http://4.bp.blogspot.com/-vOAEraOR3qk/UnnFPQFvWMI/AAAAAAAAfzE/G2oGkEvqD_A/s640/ie-vpn-pro.signed.exe.cropped.png" width="640" /></a><a href="http://3.bp.blogspot.com/-c0dOpSZjRrc/UnnFQwCxo-I/AAAAAAAAfzg/K1ohtajlpWY/s1600/ie-vpn-pro.signedrm.exe.cropped.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="32" src="http://3.bp.blogspot.com/-c0dOpSZjRrc/UnnFQwCxo-I/AAAAAAAAfzg/K1ohtajlpWY/s640/ie-vpn-pro.signedrm.exe.cropped.png" width="640" /></a></div>
<br />
Like all the other security suites, the original was flagged as a virus and removed. What happens with the transformed versions is rather more interesting. Firstly the padded version was still detected as a virus, while the randomly padded one wasn't (the random padding was longer than the non-random padding). Unique to IE the damaged signature transformation was reported as a 'corrupt or invalid' signature and treated differently to the random padding transformation. The self-signed Authenticode transformation was flagged as "not commonly downloaded" rather than a virus, and the the trusted certificate Authenticode transformation was flagged the same way but with a yellow bar rather than a red one.<br />
<br />
The diversity of messages here was surprising, clearly the signature of the file is being examined and being combined with some cloud based binary reputation system (Smartscreen filter?) before a determination is given to the user. It's worth noting that a non-malicious, unsigned, uncommon binary gave the same message as the signed (untrusted) executable and that a non-malicious, unsigned, common binary (putty.exe) gave no warning message. This means that the malicious, signed binary landed somewhere in between these two cases.<br />
<br />
<b>Part 6 - Conclusion</b><br />
<div>
First a caveat: with only a single malware sample and a small handful of security suites we can not come to any sweeping conclusions.</div>
<div>
<br /></div>
<div>
However, it looks like Windows Defender / Internet Explorer as well as Sophos take into account Authenticode signatures when scanning executables. All tested security suites seem to have very fragile signature driven engines that were defeated by almost any change to the sample but these systems are backed up by heuristic systems that are at least partially powered by a cloud based binary reputation mechanism. Windows Defender and Sophos both differentiated between untrusted Authenticode signatures and trusted signatures and Windows Defender differentiated between Authenticode signatures, a corrupted Authenticode signature, and arbitrary appended data.</div>
<div>
<ul>
</ul>
</div>
Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com3tag:blogger.com,1999:blog-1404660204966454042.post-13393234897732122472013-11-05T19:00:00.002-08:002013-11-05T21:17:04.893-08:00A Change!<a href="http://www.riotgames.com/sites/all/themes/riot/media/images/logo.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" src="http://www.riotgames.com/sites/all/themes/riot/media/images/logo.jpg" /></a>Hey everyone, just dropping in to tell you that I'm moving from <a href="http://www.google.com/security">Google</a> to <a href="http://www.riotgames.com/">Riot Games</a>. I've loved my time at Google but I'm really excited to be able to work on security in the context of online games (also I'm a huge fan of <a href="http://www.leagueoflegends.com/">League of Legends</a>). I'll still be blogging (possibly more so than before) and as always the views on this blog represent my opinions and not that of my employer.<br />
<div>
<br /></div>
Stay tuned for a new blog post very soon now, I have some notes compiled that I just have to polish.Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-64919749882581505722013-01-18T14:39:00.003-08:002018-11-23T23:03:08.637-08:00Hardcode 2013 Starts Today!<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<a href="https://3.bp.blogspot.com/-RFSHutlHKaQ/UPWAMtsNm2I/AAAAAAAACYE/qwLH-D2rhYk/s1600/HardCodeLogo-sm.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em; text-align: center;"><img border="0" data-original-height="216" data-original-width="540" height="128" src="https://3.bp.blogspot.com/-RFSHutlHKaQ/UPWAMtsNm2I/AAAAAAAACYE/qwLH-D2rhYk/s320/HardCodeLogo-sm.png" width="320" /></a><a href="https://code.google.com/p/hardcode/wiki/Hardcode2013ContestDescription" target="_blank">Hardcode 2013</a>, Google and Syscan's secure coding competition, has started! The contest information has been posted at <a href="https://code.google.com/p/hardcode/wiki/Hardcode2013ContestDescription">https://code.google.com/p/hardcode/</a><br />
<br />
From the description:<br />
<blockquote class="tr_bq">
Teams must develop a marketplace web application that allows people to organize bartering of academic goods or services in a school setting (e.g., selling used books, supplies, tutoring services). The Application should support a general marketplace where any Seller can post an Item they want to sell and any Buyer can express interest in or bid on an item. This Application does NOT include a payment transaction system; the Application connects potential Buyers with Sellers but does not perform actual payment transactions.</blockquote>
If you're a student take a look!<br />
<br />Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-37971473312207443712013-01-10T12:02:00.000-08:002013-01-10T12:02:52.031-08:00Hardcode: Google and Syscan's secure coding competitionGoogle and Syscan are running a secure coding competition with sizable cash prizes. Teams of students will build web applications of App Engine that will be judged on their features and overall security.<br />
<br />
Original post: <a href="http://googleonlinesecurity.blogspot.com/2013/01/calling-student-coders-hardcode-secure.html">http://googleonlinesecurity.blogspot.com/2013/01/calling-student-coders-hardcode-secure.html</a>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-32879911066851281312012-11-17T00:08:00.002-08:002012-11-17T00:08:22.802-08:00TextHole Source CodeA quick update.<div>
The source code to <a href="http://memeover.arkem.org/2012/09/texthole.html" target="_blank">TextHole</a> is now available from <a href="https://github.com/arkem/texthole" target="_blank">my github account</a>. </div>
<div>
<div>
<br /></div>
</div>
Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-85357731181513311092012-11-05T10:16:00.001-08:002012-11-05T10:16:37.250-08:00Tavis Ormandy's (second) Sophail paperTavis has done it again with another paper about the failings of Sophos. This time with several interesting bugs and a working exploit.<br />
<br />
Paper:<br />
<a href="https://lock.cmpxchg8b.com/sophailv2.pdf" target="_blank">Sophail: Applied attacks against Sophos Antivirus</a><br />
<br />
Full Disclosure Post (including link to exploit):<br />
<a href="http://lists.grok.org.uk/pipermail/full-disclosure/2012-November/088813.html" target="_blank">[Full-disclosure] multiple critical vulnerabilities in sophos products</a><br />
<br />Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-78793246312099561252012-09-15T15:44:00.003-07:002012-09-15T15:46:35.097-07:00TextHoleTo experiment with <a href="http://appengine.google.com/" target="_blank">Google Appengine</a> I've created a simple text repository application called <a href="http://texthole.arkem.org/" target="_blank">TextHole</a>.<br />
<br />
TextHole is a basic text repository with the following features:<br />
<br />
<ul>
<li>Anonymous uploads and read access</li>
<li>Optional Google OAuth2 authentication to allow you to delete or edit your uploads</li>
<li>A simple JSON interface makes it easy to post and download text from other sites</li>
</ul>
<div>
To download text via JSON make a GET request to http://texthole.arkem.org/download/[mesage_id]<message_id></message_id></div>
<div>
The reply will be a JSON dictionary with the following keys:</div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><b><br /></b></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">body: the text body of the message</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">editable: whether the requestor can modify the text</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;">creation: Creation time of the text</span>
</div>
<div>
<span style="font-family: Courier New, Courier, monospace;">expiry: unix timestamp of the expiry of the text</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;">message_id: the message id of this text</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">status: True if the request succeeded</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">error: If status is false more details here</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">Note: Only status and message_id fields are guaranteed</span></div>
<div>
<br /></div>
<div>
To upload text via JSON make a POST request to http://texthole.arkem.org/upload providing a JSON dictionary via the data form field. </div>
<div>
<br /></div>
<div>
Possible actions are:</div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;">New message: The body key is required</span></div>
<div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;">Delete: The delete key is required</span></div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;">Edit message: The body and overwrite keys are required</span><br />
<b style="font-family: 'Courier New', Courier, monospace;"><br /></b></div>
</div>
<div>
<div>
Request dictionary keys:</div>
<div>
<div>
<span style="font-family: Courier New, Courier, monospace;">body: the text body of the new/modified message</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">delete: the message id of the message to delete</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">overwrite: the message id of the message to edit</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">authenticated: if set attribute the new message to the user</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">expiry: number of seconds (max 1yr) the text is to be valid for</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;"><br /></span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">Notes: One of body and delete is required. Overwrite and delete require a valid cookie to be sent with the request.</span></div>
<div>
<br /></div>
Reply dictionary keys: </div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;">message_id: the message id of the new/edited/deleted text</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">status: True if the request succeeded</span></div>
<div>
<span style="font-family: Courier New, Courier, monospace;">error: If status is false more details here</span></div>
</div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;">expiry: unix timestamp of the expiry of the text</span>
</div>
<div>
<span style="font-family: 'Courier New', Courier, monospace;">user: username of the owner of the text ("None" for anonymous)</span></div>
<div>
<br /></div>
<div>
TextHole is missing the following features (maybe coming soon):</div>
<div>
<ul>
<li>An index of available texts</li>
<li>Text search</li>
<li>A javascript client library to make it even easier to integrate with TextHole</li>
<li>A way to authenticate via the JSON library</li>
</ul>
<div>
Please play around with TextHole and send me any bugs or ideas that you find. Please remember that everything in TextHole is public, I can see it, and so can everyone else. Finally, please don't use TextHole for evil.</div>
</div>
Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-71696061509290627502012-09-07T08:49:00.000-07:002012-09-07T08:49:34.626-07:00Google acquires VirusTotal<div class="separator" style="clear: both; text-align: center;">
<a href="https://www.virustotal.com/static/img/logo.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="50" src="https://www.virustotal.com/static/img/logo.png" width="320" /></a></div>
VirusTotal, the online service that will scan uploaded files against dozens of AV engines has been acquired by Google. <a href="http://blog.virustotal.com/2012/09/an-update-from-virustotal.html" target="_blank">Here's the announcement.</a> I think this is great, I'm a big fan of VirusTotal and I am looking forward to what Google and VT can come up with together.Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-63074951537873374582012-07-30T14:50:00.001-07:002012-07-30T14:50:13.476-07:00Owning Ubisoft<div class="separator" style="clear: both; text-align: center;">
<a href="http://images4.wikia.nocookie.net/__cb20110831013736/assassinscreed/images/f/f6/UPLAY_logo_-_Small.png" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="100" src="http://images4.wikia.nocookie.net/__cb20110831013736/assassinscreed/images/f/f6/UPLAY_logo_-_Small.png" width="200" /></a></div>
Tavis Ormandy is at it again, this time offhandedly revealing a drive-by code execution vulnerability in Ubisoft's Uplay platform. A malicious website could cause the Uplay browser plugin to execute arbitrary commands on the victim's computer. The attack takes advantage of a feature that allows a visited website to launch a Ubisoft game but does not check that the command that the website issues corresponds to a legitimate game. The issue has been patched in an emergency update from Ubisoft.<br />
<br />
Full details: <a href="http://seclists.org/fulldisclosure/2012/Jul/375">http://seclists.org/fulldisclosure/2012/Jul/375</a>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-21252144166175568012012-06-17T16:10:00.004-07:002021-01-24T08:03:09.274-08:00Mapping the relationship between YouTube videos<div class="separator" style="clear: both; text-align: center;">
<br /></div>
I've been playing John Robertson's YouTube choose your own adventure game <a href="http://www.youtube.com/watch?v=hvkjP6dqpfY">The Dark Room</a> and I've been having a great time. However, I need a little help navigating the room (you see, it's dark in there) and so I wrote a program to do a little cartography and create a map of the game.<div class="separator" style="clear: both; text-align: center;"><a href="https://arkem.org/ytvidmap.svg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="594" data-original-width="800" height="475" src="https://arkem.org/ytvidmap_cropped.png" width="640" /></a></div><br /><div><div class="separator" style="clear: both; text-align: center;"><br /></div><div><div>The map shows the videos that comprise The Dark Room (abbreviated here for space and to limit the spoilers) with the size of each node proportional to the number of views the video has and the colour signifying the number of outbound links from the video. The map was generated by <a href="https://github.com/arkem/ytmap/blob/master/ytvidmap.py">ytvidmap.py</a> from my <a href="https://github.com/arkem/ytmap">ytmap</a> repository and is created by processing the YouTube annotations. Sadly, the annotations aren't available from the YouTube GData API so I process the annotations with regular expressions. The map provides a huge boon in navigating The Dark Room but does not make escaping trivial (it's like John anticipated this kind of analysis).</div>
<div>
<br /></div>
<div>
After creating ytvidmap.py I realised that this approach could also be used to help me discover YouTube content by seeing who my favourite film makers and musicians linked to and in turn who they linked to. So I created <a href="https://github.com/arkem/ytmap/blob/master/ytusermap.py">ytusermap.py</a> and started by plotting the people in <a href="http://www.youtube.com/user/lindseystomp">Lindsey Stirling's</a> YouTube video social network and ended up with a giant mess of relationships that quickly got out of control. After adjusting my scripts to build in some limits I ended up with this diagram of her closest neighbors.</div><div class="separator" style="clear: both; text-align: center;"><a href="https://arkem.org/ytusermap.svg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="235" data-original-width="800" height="188" src="https://arkem.org/ytusermap.png" width="640" /></a></div><br /><div><br /></div>
<div>While not the most useful analysis tool I've ever built I've been having fun with it and you should too!</div>
<div>
<br /></div>
<div>
Check out <a href="https://github.com/arkem/ytmap">ytmap</a> on github! </div>
<div>
<br /></div>
<div>
If you like spoilers here is a complete (as far as I know) version of <a href="http://arkem.org/ytvidmap_full.svg" target="_blank">The Dark Room map</a> (use your browser's zoom function to navigate it better). </div>
<div>
<br /></div>
<div>
Finally, here is a large version of <a href="http://arkem.org/ls_big.svg" target="_blank">Lindsey Stirling's network</a> and a large version of <a href="http://arkem.org/gas_big.svg" target="_blank">GeekandSundry's network</a> (<a href="http://geekandsundry.com/" target="_blank">Felica Day and Will Wheaton's YouTube channel</a>).<br />
<br />
Edit: Viewing the images directly makes them clickable, so that they can take you directly to the YouTube user or video directly.</div></div></div>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-29719517509659740902012-03-12T18:27:00.000-07:002012-03-12T18:27:40.228-07:00Identifying computers behind NAT with plotpcapFollowing on from my last post <a href="http://memeover.arkem.org/2012/02/identifying-computers-behind-nat-with.html">Identifying computers behind NAT with pyflag</a> I've made a stand alone script <a href="https://github.com/arkem/plotpcap">plotpcap</a> that can produce similar graphs without needing to install pyflag.<br />
<br />
The results aren't as pretty and you miss out on some of pyflag's analytical tools (such as filtering streams by user agents). On the other hand you do gain the ability to filter your output by tcpdump style filter strings and with a little bit of pcap preprocessing from tshark you can perform almost all the same comparisons.<br />
<br />
plotpcap requires the python modules dpkt, pcap (from pypcap) and matplotlib. I used the versions available from the Ubuntu 10.04 repository but other versions are probably good too.<br />
<br />
Here's some output generated from the same example data as the last post:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-GWk_XzYXK8M/T16eWHhXxXI/AAAAAAAABBE/Nah6VGDc5V4/s1600/ipid.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://2.bp.blogspot.com/-GWk_XzYXK8M/T16eWHhXxXI/AAAAAAAABBE/Nah6VGDc5V4/s640/ipid.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">IPID versus Packet Number (note that without stream highlighting it gets a bit hard to read)</td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-u52a0hMw9Hs/T16ehuxhexI/AAAAAAAABBM/HEsRJ0SbaFY/s1600/ipid2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://2.bp.blogspot.com/-u52a0hMw9Hs/T16ehuxhexI/AAAAAAAABBM/HEsRJ0SbaFY/s640/ipid2.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">IPID versus Packet Number after excluding packets with TCP timestamp options (ipid2)</td></tr>
</tbody></table>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-F_vEUHea5vU/T16ewLXw3nI/AAAAAAAABBU/H8XhUMzWrOo/s1600/tcptsval.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://3.bp.blogspot.com/-F_vEUHea5vU/T16ewLXw3nI/AAAAAAAABBU/H8XhUMzWrOo/s640/tcptsval.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">TCP Timestamps versus Packet Number</td></tr>
</tbody></table>
If you wanted to do some of the tricks from the last post you can apply wireshark display filters to the pcap and then run it through plotpcap. For example:<br />
<br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">tshark -r test.pcap -w test_chrome.pcap -R "http.user_agent contains Chrome"</span><br />
<span style="font-family: 'Courier New', Courier, monospace; font-size: x-small;">python plotpcap.py test_chrome.pcap number ipid</span><br />
<span style="font-family: 'Courier New', Courier, monospace;"><br /></span><br />
<span style="font-family: inherit;">Produces something like:</span><br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-i5o0OPxjCp8/T16hwicTWSI/AAAAAAAABBc/J9u67yKD-DQ/s1600/ipid_chrome.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="http://3.bp.blogspot.com/-i5o0OPxjCp8/T16hwicTWSI/AAAAAAAABBc/J9u67yKD-DQ/s640/ipid_chrome.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">IPID versus Packet Number after matching the wireshark display filter "http.user_agent contains Chrome"</td></tr>
</tbody></table>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com4tag:blogger.com,1999:blog-1404660204966454042.post-13762456024497610462012-02-20T01:55:00.000-08:002012-03-12T18:29:03.329-07:00Identifying computers behind NAT with pyflagI've been a bit busy recently as I'm preparing to move across the world to the US to work at a <a href="http://www.google.com/">small Internet company</a> in the SF Bay Area. In the mean time though my current employer has been kind enough to let me contribute back some of the code we have written for the <a href="https://github.com/arkem/pyflag">pyflag</a> project (the link goes to my github page which has a fork of the project as the upstream site <a href="http://pyflag.net/">pyflag.net</a> is down right now). Update: An alternate version (without the feature described below) <a href="http://code.google.com/p/pyflag/">is available on google code</a><br />
<br />
The new features centre around identifying computers that are all lumped together behind a network address translation gateway (NAT). The idea is if you can identify the computers behind the NAT gateway you can attribute traffic to a specific system rather than only down to the network itself. The implementation is some visualisation tools in pyflag that allow you to plot certain packet headers fields against packet numbers or time.<br />
<br />
Here's an example:<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-c1umPXBPCMU/T0H3YxWGOeI/AAAAAAAAA_0/ptp1Yg_WsQk/s1600/all+the+traffic.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="340" src="http://2.bp.blogspot.com/-c1umPXBPCMU/T0H3YxWGOeI/AAAAAAAAA_0/ptp1Yg_WsQk/s640/all+the+traffic.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">IPID field plotted against PCAP packet number</td></tr>
</tbody></table>
The plot takes the IP Identification field from the IP header and plots it sequentially against the PCAP packet number (pyflag also supports plotting against time). It looks like a big mess but you can see some lines and maybe some patterns in there. The IPID field is used to associate fragmented packets together for reassembly and it is generally left untouched by NAT gateways. Usefully different networking stacks have different strategies for picking IPID values.<br />
<br />
In my anecdotal (non-scientifically determined) experience:<br />
<br />
<ul>
<li> Windows machines start at 0 when the computer is booted and increment for each packet sent up until 2^16 and then start again. In some cases it seems to wrap at 2^15 which to me suggests a signed integer problem but I haven't conclusively figured out on what versions it happens on. Additionally, I've read (but not seen) that some versions of Windows send the field in host order rather than network byte order.</li>
<li>Linux machines pick a random number for the start of the connection and then increment the value for each subsequent packet of the connection. I've heard (but again not seen) that packets with the Don't Fragment bit set get their IPID set to 0 on Linux.</li>
<li>BSD machines (including Mac OS X) pick a random number for every packet.</li>
</ul>
<div>
So looking back at our example we can see a haze of small lines and also a couple of longer lines which suggests that we might be looking at one or more Linux boxes along with one or more Windows boxes. To test this theory I looked for any user-agent strings in web traffic and found the following:</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-U3NiM7ReItE/T0H6S2ishvI/AAAAAAAAA_8/swUaLXgf9vA/s1600/user+agents+cropped.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="564" src="http://4.bp.blogspot.com/-U3NiM7ReItE/T0H6S2ishvI/AAAAAAAAA_8/swUaLXgf9vA/s640/user+agents+cropped.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">User-Agent strings present in the sample PCAP file</td></tr>
</tbody></table>
<div>
Based on those user agent strings it looks like there is at least one Ubuntu system and one Windows system. Also of note is the presence of Java user agent strings as well as Transmission (the Ubuntu Bittorrent client).</div>
<div>
<br /></div>
<div>
If we revisit our previous IPID plot and tell pyflag to colour all the Chrome/Windows user agent string related streams blue we get the following:</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-HFBIeYT_VWc/T0ID9P0rItI/AAAAAAAABAE/FbnVUy9CqXw/s1600/windows+chrome+plot+2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="342" src="http://3.bp.blogspot.com/-HFBIeYT_VWc/T0ID9P0rItI/AAAAAAAABAE/FbnVUy9CqXw/s640/windows+chrome+plot+2.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">IP ID versus PCAP number with Chrome on Windows streams highlighted</td></tr>
</tbody></table>
<div>
From this it becomes clear that there are two distinct lines of IPID growth which implies that behind this NAT gateway are two Windows systems, one which was active for longer and even sent enough packets that the IPID value wrapped. Knowing the shape of these lines means that you can associate other traffic (perhaps traffic with no distinguishing application layer features such as encrypted streams) to a specific computer and any metadata gleamed from other application protocols (like HTTP). </div>
<div>
<br /></div>
<div>
To make this even clearer there's another header field to consider, this time in the TCP header. There is an optional header in TCP called the timestamp value (defined by RFC1323) which is used to measure packet round trip times. By default Windows systems omit this value while most other systems include it (I've read that Windows can be configured to send timestamps and that in some cases will use timestamps if the client connecting to it uses timestamps). This means that if we exclude packets that have a TCP timestamp we should be left with all Windows traffic (assuming we exclude non-TCP traffic as well).</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-olCh29qLt9k/T0IGGUYnWqI/AAAAAAAABAM/sl6DlY_9r78/s1600/windows+chrome+plot+3.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="342" src="http://2.bp.blogspot.com/-olCh29qLt9k/T0IGGUYnWqI/AAAAAAAABAM/sl6DlY_9r78/s640/windows+chrome+plot+3.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">IPID versus PCAP number for Chrome user-agents, minus packets that have a TCP timestamp</td></tr>
</tbody></table>
<div>
After excluding packets with the TCP timestamp option set most of the background packets have been excluded. The remaining packets that don't fall on the lines are likely parser failures or packets generated by a Linux box that do not have a timestamp value for one reason or another (more investigation is required).</div>
<div>
<br /></div>
<div>
So we're convinced that there are two Windows system on the network and some yet to be determined number of Linux systems, if we change our filter to highlight Firefox on Linux and then plot IPID we get something that looks like this:</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://4.bp.blogspot.com/-idWEbYDnp4k/T0IH_EUlkWI/AAAAAAAABAU/QxuifraQvcI/s1600/linux+web+traffic.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="342" src="http://4.bp.blogspot.com/-idWEbYDnp4k/T0IH_EUlkWI/AAAAAAAABAU/QxuifraQvcI/s640/linux+web+traffic.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">IPID versus PCAP number for Firefox sessions on Linux</td></tr>
</tbody></table>
<div>
The things to note here is that the IPID values change dramatically between connections, also that in general HTTP traffic seems to be in the minority of the non-Windows traffic and finally that we're no closer to determining how many Linux systems are present. However, if we consider the TCP timestamp field for a moment we learn that it's generally determined as:</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://3.bp.blogspot.com/-_8geOqFETcI/T0IJKZgSSkI/AAAAAAAABAc/geyVTGM5YcI/s1600/timestamps.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="54" src="http://3.bp.blogspot.com/-_8geOqFETcI/T0IJKZgSSkI/AAAAAAAABAc/geyVTGM5YcI/s320/timestamps.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">From: <a href="http://www.slideshare.net/gwicherski/identifying-hosts-with-natfilterd">Identifying hosts with natfilterd</a></td></tr>
</tbody></table>
<div>
The interesting part in this case is that wallclock - boottime should be unique among the hosts that use the TCP timestamp option and it should increment in a predictable fashion. So if we graph the TCP timestamp value of packets versus their PCAP number we get:</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-GBK7r6AV6PU/T0IJ2YJPNzI/AAAAAAAABAk/DGWnjFPbOAs/s1600/linux+web+traffic+tsval.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="342" src="http://2.bp.blogspot.com/-GBK7r6AV6PU/T0IJ2YJPNzI/AAAAAAAABAk/DGWnjFPbOAs/s640/linux+web+traffic+tsval.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">TCP Timestamp value versus PCAP packet number (Firefox/Linux traffic highlighted)</td></tr>
</tbody></table>
<div>
Again we can see that the Firefox traffic accounts for only a minority of packets and we also see that there're two distinct lines for the first half on the plot. These two lines suggests that there are two Linux systems and the line fragment at the end probably represents a reboot (and not wrapping because the timestamp values are 32 bit numbers and the values we see are around 2^18 at their highest) of one of the systems or the appearance of a new one.</div>
<div>
<br /></div>
<div>
So at this point I'm convinced that there are two Linux systems and two Windows system and that most of the Windows packets are HTTP traffic (using Chrome) and that while there is HTTP traffic it accounts for only a small amount of the Linux related packets. For the remainder of the Linux traffic I'd guess that at least one of the systems is transferring files using BitTorrent based on the Transmission user-agent that was present before. Maybe if we plot the traffic with the Transmission user-agent we'll be able to tell which computers were running BitTorrent:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-ZtUKPUdU3h8/T0IQJyNpc8I/AAAAAAAABA0/iXWKa0Nk7vo/s1600/transmission.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="342" src="http://1.bp.blogspot.com/-ZtUKPUdU3h8/T0IQJyNpc8I/AAAAAAAABA0/iXWKa0Nk7vo/s640/transmission.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">TCP Timestamp versus PCAP Packet Number for the user-agent "Transmission"</td></tr>
</tbody></table>
<div>
At first this looks good, the line with the lower timestamp values is associated with Transmission and the higher one is not. Unfortunately this plot is ambiguous because the third line section is also associated with Transmission traffic and that line could easily belong to the top line section (after a reboot). If instead we ask pyflag to generate a table with only traffic that is not to or from ports 80 or 53 (to eliminate HTTP and DNS) we're left with a lot of connections between high ports transferring lots of encrypted (looking) data to our NAT gateway address which fits the hypothesis of BitTorrent traffic. When we plot the timestamp values again and highlight any packet from our Not-HTTP/Not-DNS table we get the following:</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://2.bp.blogspot.com/-lxQJnMgc6KA/T0ITcUpv_aI/AAAAAAAABA8/Wy0ShSYreCM/s1600/non-http+non-dns.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="342" src="http://2.bp.blogspot.com/-lxQJnMgc6KA/T0ITcUpv_aI/AAAAAAAABA8/Wy0ShSYreCM/s640/non-http+non-dns.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">TCP Timestamp versus PCAP number with non-HTTP/non-DNS traffic highlighted</td></tr>
</tbody></table>
<div>
At this point I'm reasonably confident that both the observed Linux hosts are downloading files over BitTorrent once I combine this plot with some analysis of the ports / stream sizes seen while I'm equally convinced that the Windows systems are not using BitTorrent or at least that there isn't a significant level of BitTorrent traffic observed during this packet capture.</div>
<div>
The above little demo is contrived but I have found that this kind of analysis can be really useful in characterising the use of a network. This example was constructed from 5 virtual machines, 2 running Windows XP, 2 running Ubuntu 10.04 and a NAT gateway running Ubuntu 10.04 and using iptables/netfilter to do the NATing. Also, just in case you were wondering the Windows machines were watching youtube (in particular <a href="http://www.youtube.com/watch?v=wZZ7oFKsKzY">nyan cat</a> and <a href="http://www.youtube.com/watch?v=WG60-0tp5sU">techno viking</a>) while the Ubuntu systems were each using BitTorrent to download ubuntu images (12.04 alpha for different architectures). </div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Future Work</div>
<div>
<ul>
<li>Spring cleaning of the pyflag source (it's a little annoying to build and use right now)</li>
<li>More options on what to graph (maybe a system for generically plotting table information)</li>
<li>Ability to choose what to highlight based off the reverse side of a stream</li>
<li><strike>Implementing a minimal version of this visualisation outside of pyflag</strike> Done! <a href="http://memeover.arkem.org/2012/03/identifying-computers-behind-nat-with.html">Identifying computers behind NAT with plotpcap</a></li>
</ul>
<div>
<br /></div>
<div>
Related Work and Further Reading</div>
</div>
<div>
<ul>
<li><a href="http://nmap.org/book/osdetect-methods.html">nmap book - os detection</a></li>
<li><a href="http://lcamtuf.coredump.cx/p0f3/">lcamtuf's p0f3 fingerprinting tool</a></li>
<li><a href="http://www.slideshare.net/gwicherski/identifying-hosts-with-natfilterd">Georg Wicherski's Identifying hosts with natfilterd</a></li>
<li><a href="https://www.cs.columbia.edu/~smb/papers/fnat.pdf">Steven M. Bellovin's A Technique for Counting NATed Hosts</a></li>
<li><a href="http://www.phrack.org/issues.html?id=3&issue=63">Elie aka Lupin TCP Timestamp to count hosts behind NAT (Phrack 63)</a></li>
</ul>
<div>
<br /></div>
<div>
Update:</div>
</div>
<div>
Now that I've got the links handy I thought I'd also point at Michael Cohen's work. Michael is one of the authors of pyflag (project lead is probably a better description), and it's his ideas and that lead to the implementation of IP ID processing in pyflag.</div>
<div>
<ul>
<li><a href="https://docs.google.com/viewer?a=v&pid=explorer&chrome=true&srcid=0B9hc84IflFGbODRjMzc4ZjgtNWJiNS00NWRlLWJhYjQtZTk3Mjg1ODc0ODA3&pli=1#">Source attribution for network address translated forensic captures</a></li>
<li><a href="https://docs.google.com/viewer?a=v&pid=explorer&chrome=true&srcid=0B9hc84IflFGbZTkyODRjYzUtNzNiYi00MzNhLWI3OTEtM2M4ZWRkMjQzZDk4&pli=1#">Network forensics - A Practical Introduction</a></li>
</ul>
</div>
<div>
<br /></div>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-53928998361091931452012-01-02T18:09:00.000-08:002012-01-02T18:09:50.213-08:00Yet Another First Ascension Post<div class="separator" style="clear: both; text-align: left;">
I was going through the pages of an old defunct blog of mine and I saw this image and thought that I would repost it for old times sake. This is one of my proudest computer gaming moments of all time (from October 2009).</div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://arkem.org/nethack/score.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="534" src="http://arkem.org/nethack/score.png" width="640" /></a></div>
<br />Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com2tag:blogger.com,1999:blog-1404660204966454042.post-74540477827037023482011-09-06T05:44:00.001-07:002011-09-06T05:46:17.413-07:00Something you should know about talloc<a href="http://talloc.samba.org/talloc/doc/html/index.html">Talloc</a> is an excellent memory management system for C that provides hierarchical memory pools with other cool tricks like destructors. It's written by <a href="http://en.wikipedia.org/wiki/Andrew_Tridgell">Tridge</a> for Samba and I really like it. If you are writing a complex system in C you could do worse than to replace your calls to malloc with calls to talloc.<br />
<br />
So that's talloc, but the thing you really should know about talloc is right there at the bottom of the project page. In particular:<br />
<span style="font-family: 'Courier New', Courier, monospace;"></span><br />
<div><span style="font-family: 'Courier New', Courier, monospace;"><span style="font-family: 'Courier New', Courier, monospace;"><br />
</span></span></div><span style="font-family: 'Courier New', Courier, monospace;">when using talloc_enable_leak_report(), giving directly NULL as a parent context implicitly refers to a hidden "null context" global variable, so this should not be used in a multi-threaded environment without proper synchronization.</span><br />
<div><span style="font-family: 'Courier New', Courier, monospace;"><br />
</span></div><div><span style="font-family: inherit;">I've spent many days recently hunting down a bug, the bug would have been much easier to find if I had read the above line. Suddenly I was sharing contexts all over the place and <i>very very rarely</i> there'd be a synchronization problem that would lead to a null pointer deref. </span></div><div><span style="font-family: inherit;"><br />
</span></div><div><span style="font-family: inherit;">By the way talloc_enable_leak_report() is an excellent feature of talloc. Excellent. </span></div>Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0tag:blogger.com,1999:blog-1404660204966454042.post-59350502038529436272011-08-08T04:43:00.000-07:002013-11-07T18:11:39.807-08:00Authenticode and Antivirus Detection part 2<b>Update Nov 2013: </b>Another follow up post: <a href="http://memeover.arkem.org/2013/11/authenticode-and-antivirus-detection.html">Authenticode and Antivirus Detection Revisited</a><br />
<br />
After Shane's comments on the <a href="http://memeover.arkem.org/2011/08/authenticode-and-antivirus-detection.html">Authenticode and Antivirus Detection</a> post I thought I'd run some more tests. I wanted to try and figure out how much of the observed detection difference were because some extra bytes had been added and how much was due to special handling of signed binaries.<br />
<br />
I found an archive of malware online and created four sets of samples. Set one was the malware without any changes, set two was after the binaries had been signed with the TEST1 certificate, set three was signed with a TEST2 certificate that was similar to TEST1 but was only valid from 1975 - 2009 and set four had a random blob of 32 bytes appened to the end. Using the VirusTotal API and <a href="http://www.bryceboe.com/2010/09/01/submitting-binaries-to-virustotal/">Bryce Boe's python script</a> I ran each of the sets against the VirusTotal antivirus suite.<br />
<br />
The resulting statistics are <a href="http://arkem.org/amag_results_100_sorted.txt">here</a>, showing the number of AV positives, the format is:<br />
<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;"> "HASH [SET1, SET2, SET3, SET4] [SET1 - SET2, SET1 - SET4]"</span><br />
<br />
And here are the first 10 entries (ordered by decreasing "SET1 - SET2" value):<br />
<br />
<pre style="white-space: pre-wrap; word-wrap: break-word;"><span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">DB1D5E...34573 ['28', '10', '16', '22'] ['18', '6']
FDFB86...1FE0C ['34', '18', '18', '27'] ['16', '7']
6D48A7...F4880 ['36', '20', '22', '33'] ['16', '3']
CA9C3E...ED072 ['31', '16', '16', '23'] ['15', '8']
8798FA...8755B ['35', '20', '20', '32'] ['15', '3']
1011ED...0DB18 ['35', '20', '20', '33'] ['15', '2']
DA01D0...C899D ['31', '17', '16', '28'] ['14', '3']
CC3B7D...228D1 ['37', '23', '23', '34'] ['14', '3']
CADD90...CE9C4 ['35', '21', '21', '31'] ['14', '4']
B6BBE8...8CD10 ['32', '18', '18', '29'] ['14', '3']</span></pre>
<br />
<span class="Apple-style-span" style="font-family: inherit;">General observations:</span><br />
<ul>
<li><span class="Apple-style-span" style="white-space: normal;"><span class="Apple-style-span" style="font-family: inherit;">Adding either an authenticode signature or random data would defeat several engines</span></span></li>
<li><span class="Apple-style-span" style="white-space: normal;"><span class="Apple-style-span" style="font-family: inherit;">Very rarely would the signing certificate's validity influence the score</span></span></li>
<li><span class="Apple-style-span" style="white-space: normal;"><span class="Apple-style-span" style="font-family: inherit;">For some reason adding the random data occasionally resulted in more signatures being hit and considering that the same data was added to each sample I'm not sure what happened there.</span></span></li>
<li><span class="Apple-style-span" style="white-space: normal;"><span class="Apple-style-span" style="font-family: inherit;">This test primarily tests the AV signature engines and not their runtime or heuristic scanners</span></span></li>
<li><span class="Apple-style-span" style="white-space: normal;"><span class="Apple-style-span" style="font-family: inherit;">The VirusTotal API limit of 20 requests each 5 minutes sounds like a lot until you run tests like this.</span></span></li>
</ul>
<div>
Really what I've learnt from this is that AV signatures are even more fragile than I realised. To get a proper look at how AV treats authenticode signed binaries I think I'd need to evaluate all of each AV's modules and not just the signature engine.</div>
Arkemhttp://www.blogger.com/profile/05047833961750578893noreply@blogger.com0