Boldlife Episode 7
On the latest episode of BoldLife, we’re speaking with Aaron Campbell, the WordPress Security Team Lead. We discuss security in the open source space, and Aaron’s experience as a fulltime WordPress contributor.
Show Notes / Transcription:
Mike “demo” Demopoulos
host & boldgrid evangelist
A longtime lover of Open Source Software, Mike “Demo” Demopoulos currently works at BoldGrid (a WordPress Site Builder) as an Evangelist. He has spoken at numerous open source events around the world. Mike is also a contributor to Huffington Post as well as other publications. In addition, he volunteers as Treasurer for Open Source Matters.
WordPress Security Team Lead
Aaron is the WordPress Security Team lead, has been a regular contributor to WordPress for more than a decade, and is currently funded by GoDaddy to work full time on the WordPress open source project. He has over eighteen years of web development experience and worked with clients ranging from small local businesses to Google, Yahoo, Disney, and Harvard. He’s been called both a coffee snob and a beer snob, but considers both to be compliments. When not buried in code, he enjoys spending time with his wife and son, riding his motorcycle, and reading sci-fi/fantasy books.
MikeDemo: Hello! Welcome to the seventh episode of the BoldGrid BoldLife Facebook Live Show. My name is Mike Demo, and I am pleased to be joined with Aaron Campbell from the WordPress Core Security Team.
Aaron Campbell: Hey, how’s it going?
MikeDemo: Good, how are you doing?
Aaron Campbell: I’m doing well.
MikeDemo: Welcome to Minnesota.
Aaron Campbell: Thank you. Yeah, we’re split screen on here, but actually sitting next to each other in person, so that’s kind of cool.
MikeDemo: Yeah, it’s the first time we’ve done one of these shows in person, so it’s kind of exciting to actually be able to see the person I am talking to, so …
Aaron Campbell: Yeah, I’m your first, awesome. Well, happy to be.
MikeDemo: So, yeah, for those of you that don’t know what you do, kind of explain what your role is in the project.
Aaron Campbell: Yeah, so, I lead the security team, which basically amounts to a lot of, I guess, coordination, and running meetings, and that kind of stuff, but, you know, I’m trying to focus on keeping almost 30% of the web secure now. I’ve got a whole group that helps me, thankfully that does not rely just on me, but, yeah, that’s what I try to do.
MikeDemo: Awesome, and you are fortunate enough to be able to do this full time, how does that work?
Aaron Campbell: Yeah, so I am funded by GoDaddy to do this full time, which is pretty fantastic. They hired me almost a year and a half ago now to work full time on the WordPress Open Source Project, just kind of given the goal to make WordPress better, and now I do that by leading the security team. I think that there are certain positions in a project like this that are less fun to do with volunteer time, and something that lends itself to a person that can focus on it full time, so that’s kind of how I went the direction that I went.
MikeDemo: And you’re … Thanks to GoDaddy, then, you’re able to devote full time focus to the project that you might not be able to do if you were just a traditional volunteer with maybe a freelance job, or your own company or something.
Aaron Campbell: Right, I’ve been volunteering time to work on WordPress for a very long time, but GoDaddy has given me the chance to spend full-time hours on it, which is pretty fantastic.
MikeDemo: That’s great. That’s really great. So, looking back at kind of … How did you get started with WordPress originally? Was it your kind of first CMS? What’s kind of your history to kind of get to the point of where you are now?
Aaron Campbell: It wasn’t my first CMS. I started doing web development about 18 years ago or so, so WordPress didn’t exist.
Aaron Campbell: But WordPress came about, and it had been around for maybe a year and a half or two years when I started to look at it for potentially doing client projects. I wanted something that I could give to a person when they were done, or when I was done building it, and they would be able to use it to continue to generate content for the site, and at the time WordPress seemed like the best option.
When I look back now at what it looked like then and how it worked then, I’m impressed that clients were able to do that, because it was definitely much harder to use back then. But yeah, I started out just using it to build things, and then, after I’d been doing that for a year or two I came across a problem that I wanted to try to fix in WordPress Core, and kind of figured out how to contribute a patch, and as soon as that code made it into the project I was hooked.
I went, oh, I can make this better by just giving them code now and then? And I just sort of have been contributing to literally every single release since then. That was, like, 11 years ago now.
MikeDemo: Okay, and when did you start getting involved with the security team at WordPress?
Aaron Campbell: You know, someone else asked me that relatively recently and I’m a little foggy on that, honestly. It’s been a while. It’s probably been, you know, six or eight years. It was … There was a point at which we didn’t have much of a formalized security team, and I just helped out because I was one of the people that helped out with everything in the project because we were smaller back then.
At some point it kind of formalized, and I didn’t really have time to be a part of it, so I wasn’t, but sometime about maybe six or eight years ago I joined just to help out when I could, and then I took over leading it about a year and a few months ago, little over a year ago.
MikeDemo: Okay, excellent. And I have seen some blog posts and some, when you became the security lead, with the title security czar. Are you fond of that title?
Aaron Campbell: You know, they didn’t give me a scepter, so, I figure, no scepter, no czar. I’m not a huge fan of that. I think that it’s definitely a huge team effort to secure a project the size of WordPress, so I prefer talking about it in terms of the team rather than czar, but that was how the announcement post was worded, so …
MikeDemo: Well, yeah, like with a lot of open source projects, it’s all just a group of volunteers that are-
Aaron Campbell: Yeah.
MikeDemo: … pulling the rope in the same direction for this thing that they love that helps them, you know, make money, but also, you know, fosters friendships and things, too.
Aaron Campbell: Yep, for sure.
MikeDemo: Excellent. So, looking at security in WordPress, how has the landscape changed now versus what, you know, the Internet security was a while ago, because you hear all these people that are like, “My site’s not a target, I just sell flowers,” or, what, “Just a blog,” so, how has the landscape changed over the years?
Aaron Campbell: I mean, I distinctly remember building sites for the Internet and not thinking about security at all. Like, that was not a thing that crossed my mind even as I build sites for corporations and such. It just wasn’t … People didn’t worry about it very much, say, 12 or 15 years ago.
It’s hugely different now, largely because there’s a lot more information out there. People have found ways to make money either off of getting that information or just compromising sites and putting ads and stuff on them, and anywhere that there’s money to be made like that there’s going to be a lot more effort put into it.
And, so, now we’re seeing … Everyone pictures their sites being hacked by a person sitting at a computer actively trying to get into their site, you know, something a little-
MikeDemo: Yeah, like, one guy trying to go after a very specific target, like, you know, I don’t like your blog so I’m going to sit here and break your blog.
Aaron Campbell: Right.
MikeDemo: That’s what people think of when they-
Aaron Campbell: Right. I mean, literally, like, the movie Hackers, right? I love that movie, I have to say, but that it’s not like that anymore. Instead, most of the hackers are not actively hacking into a site, they’re writing a program, writing a script that then crawls the web and breaks into hundreds, or thousands, or hundreds of thousands, or millions of sites, and it’s a very automated process.
It doesn’t care who you are or, you know, what your site does, or how many visits your site gets, or anything, it’s just wondering, is your site easy to break into, because if it is, I’m going to use it for my own purposes. And, so, every single person is now a target, because no one’s just picking targets, they’re just crawling the web and everyone is a target.
And, so, it’s changed the landscape dramatically. Like, there aren’t sites that aren’t attacked. It just doesn’t happen anymore.
MikeDemo: Sure. I was at a hosting conference, and the big thing a lot of these hosters, especially shared hosts, keep … Was talking about is that in the new, you know, cryptocurrency, you know, BitCoin, and mining, and things like that, a lot of people are hacking into sites just to use the resources to mine cryptocurrency.
And the site owner might not even have any … You know, be aware that it’s happening, but that can really, you know, kill the resources of that entire server, or that VPS, or that shared account, which … And then can also cost power, and it’s causing a lot of issues for hosters in some cases, I’ve read.
Aaron Campbell: Hosts-
Aaron Campbell: Hosts are definitely frustrated with this. I mean, I definitely hear about this from some of the GoDaddy folks for example, that definitely see a lot of this, and now they’ve kind of taken it one step further, right? Servers seem like they would have a ton of power for mining cryptocurrency, and there are some kinds of cryptocurrency that can be mined pretty well on web servers, but actually personal computers tend to be better because they have a GPU, a graphics processing unit, which tends to be better at mining cryptocurrencies.
MikeDemo: Through the browser agent.
Aaron Campbell: Right.
Aaron Campbell: So, they’re … Through the browser they’re mining currency, which is … It’s all kinds of frustrating, right?
Aaron Campbell: It’s another way of them to make money. The main reason it’s frustrating right now is because it’s new and different. They’re coming up with new and different ways of doing this. You know, they’re using stuff through the browser to use the person … Like, this isn’t a thing that we’ve seen up until we had cryptocurrency, right?
And hackers, love it or hate it, are very good at what they do. Like, these people are very intelligent, very smart, capable of doing a lot of things that you look at and you go, wow, that was … That was brilliant. I can’t believe they thought of that, and did that.
And, so, when they find something new and effective like this, and everybody’s kind of scrambling to find ways to stop it, it’s kind of the rough part of the curve for us. We’re constantly chasing each other back and forth. We’re blocking them, they’re finding something new. Maybe we’re coming up with new ways and trying to block them before they even get there, but I think that a lot of people feel a little behind the curve on this one.
MikeDemo: Sure. We’ll talk about what, like, an end-site owner can do in a minute, but from the project level, how do you work with other open source projects with the security landscape, like when, for example, if you guys discover there’s a PHP exploit, or … How do you go about responsible disclosure with these other projects? And, you know, even frameworks that are … You know-
Aaron Campbell: Yeah.
MikeDemo: … that are out there.
Aaron Campbell: So, this is one of the hardest parts about security in general, but especially for a project like WordPress, right? We … You know, you mentioned, what if there’s an issue with PHP itself, well, that’s going to affect not just WordPress but probably also Drupal, and Joomla, and all these other projects. Maybe it’s an issue that’s in a library that WordPress uses, and maybe other projects use it too.
One of the things that we, as a project, try to do, or we as a security team try to do is build relationships with all these other projects with the … All the libraries that WordPress uses. You know, we build relationships with those maintainers, with the other projects. I was just at DrupalCon and got to spend time with Drupal’s security team, so that when these things come up we can all work together, fix things in a coordinated way, release in a coordinated way, possibly even work together to figure out what the best solution is.
Fixing things in WordPress Core that only affect WordPress Core, especially if they don’t affect plug-ins, or themes, or … Like, that’s actually kind of the easy part. It’s when things touch lots of projects, or could affect thousands of plug-ins, or sites that aren’t WordPress, that definitely gets a lot more challenging.
MikeDemo: Okay. So, if you’re a site owner and you just, you know, let’s say you have a WooCommerce site and you sell crafts or something, what can you do to make sure that your site is secure? Because there are … You know, there are third-party plug-ins, there’s web application firewalls, like, through Sucuri, or SiteLock, or some of the other, like, [inaudible 00:13:48], some of the other tools.
What can a … Just the average site owner do at bare … Like, I know there’s layers to it, but at a bare minimum, what should every WordPress site owner do to protect their site, and then what would you recommend if they were going to layer on top of that?
Aaron Campbell: Yeah, I mean, my go to answer for this, generally, is keep everything up to date. Like, that’s kind of the top thing, and we try to help WordPress users do this through our auto update system. It’s one of our most powerful tools for helping other people keep their sites secure is that when we find a vulnerability, or one’s reported to us, we go through the process of fixing, testing, releasing, and then we push it out to sites very quickly and auto update as many sites as possible with the security fix.
But keeping your … Not just your site, but also your plug-ins, and your PHP, your database, your MySQL or whatever you’re using, NGINX or Apache for your web server, like, all this needs to be up to date as well, which means you either need to do it, or you need to pick a web host that’s going to do it for you.
But keeping things up to date is really important, and that’s tied real closely to the next biggest thing that I try to point out, which is that the vast majority of times it’s the human aspect that is the weakest link in security on most sites. It’s poor passwords, or poor password practices. It’s the people that are not being careful with, you know, leaving something, say, an FTP session logged in on some computer, or using sketchy WiFi somewhere.
Teaching all your users how to keep themselves secure is probably the next best thing that you can do for keeping your whole site secure.
MikeDemo: Sure. So, you mentioned passwords and good password practices, what about, do you think, two-factor authentication, just for security in general, is something that the average person should do? Like, I … Personally, I use a UBKey, which is a hardware two-factor token.
Aaron Campbell: Yep.
MikeDemo: But, I [inaudible 00:16:18] I was at an agency once, we had 150 sites, and we required every client to have a UBKey.
Aaron Campbell: Yeah.
MikeDemo: They hated us for it.
Aaron Campbell: Absolutely. So, let me back up just a second and touch on passwords again, just because when I say good password, not everybody knows what that is, right?
Aaron Campbell: To me, a good password is three things, it’s long, it’s random, and it’s unique. Long, I don’t mean eight characters, I mean, like, 20 plus. Random, I don’t mean it’s something that, you know, some random chunk of text from a poem that I use, but I mean randomly generated. And unique meaning it’s only ever used in one place. So, to do that you pretty much need a password manager.
So, that’s the best … Like, that … You can’t have good password practices online without a password manager, and when I try to teach general users to do this, there’s a lot of griping, and complaining, and I don’t want a new tool, and why can’t I just use the same password that I use on every website, always, and it’s my dog’s name and no one’s going to know that except friends and family anyway, right? These are all things that I’ve actually heard from people.
But, in general, once you get a person set up on a password manager, like 1Password, or LastPass, or something like that, it takes them a little bit of time to learn it, and then it’s not slower. It may actually speed up their processes, make things easier for them.
MikeDemo: Because it might auto-fill and-
Aaron Campbell: Right.
Aaron Campbell: Having said that, two-factor is so much better than even a good password, right? It’s like a good password that you change every 30 seconds, which is phenomenal. Like, that’s kind of the next biggest weakness of passwords is that we don’t change them very often. I mean, everyone’s got a password that they’ve had for at least a year, right? So, if that password was snagged off coffee shop WiFi, or airport WiFi, or something like that, it’s probably good to use for a very long period of time, whereas if they pulled that and your two-factor code, it’s good for a couple minutes, max.
So, it’s fantastic, but people … Again, it’s a new tool to learn, right? They need to use an app on their phone, or something like a UBKey, something they need to keep with them, and it doesn’t get better. It’s always going to take an extra 10 seconds to 30 seconds to log in, and as much as I can say that that 10 to 30 seconds adds so much security that it’s hard to quantify it, like, it’s not twice as secure, or 10 times as secure, it’s way, way, way more secure, people still gripe about it a lot, and I don’t know why.
MikeDemo: Yeah, but, I mean, you know, if someone’s on their phone, and they are trying to buy something on their eBay app, and then it’s like, oh, we’re going to send you the text message, and then you have to do all … You know, jump-
Aaron Campbell: Yep.
MikeDemo: … through all these hoops, I think people are used to instant gratification in technology.
Aaron Campbell: And there’s a balance, like, with security … So, like, you can make all things perfectly secure in theory, right? You could … I mean, you would have to, like, delete your site, or disconnect it from the Internet or something to make it perfectly secure, but there’s always going to be a balance. Like, security … Like, what our team does, what the security team does, is finding the right balance between absolutely secure and usable.
And to kind of highlight this, a bank vault is extremely secure, and you also want your house to be secure, but you don’t want to live in a bank vault. You want to be able to be carrying a load of groceries and open your door. You don’t want someone else to just be able to walk into your house and take all your stuff, but there’s some sort of balance between security and ease of use that you find, and that is different for your house than it is for a bank vault.
Aaron Campbell: And it may be different for a website that’s processing thousands of dollars, or tracking peoples’ private information, versus my parents’ blog, but everyone has to find that balance. I, personally, think that for most people, that is at least a very strong password, but for most people I think that if you could get used to a two-factor auth, and that is still within that balance, but maybe not for everybody. Maybe you’re working with very beginners on your site, or people that don’t have access to that kind of tech all the time where it really is much more an issue for them.
And, so, everyone has to find that kind of balance, but you need to take the time to actually asses it, and weigh it out, and say I’m purposely choosing this balance point. This is enough security for us without losing the ease of use, but you need to make that a conscious decision and take the time to think it through.
MikeDemo: But, like you said, once people are set up on, let’s say, a password manager, they kind of get used to it. I’ll give an example, my wife. So, I was like, hey, we’re getting LastPass, and you’re going to get a UBKey, and she hated me for, like, two months. Now she hates me for different reasons, but not because of the UBKey, and she uses it for everything. And then she just got a new computer, and there’s only USB C on it, and the first thing she did is, she got her computer, she, like, messages me, and I was at a work camp, “I need a new UBKey, because I need a USB C UBKey,” and I’m like, “That is amazing.”
Aaron Campbell: Not, I no longer need this UBKey, I’m throwing it out, but I need the proper replacement [crosstalk 00:22:40]
MikeDemo: I want a USB C. So, now they make the ones that are flush with the port, so it can just stay in there and [inaudible 00:22:44] but, like, yeah, we just upgraded our WiFi network, [inaudible 00:22:49] now has their own router that does the packet sniffing and stuff like that in the router itself, it uses their DNS and their VPN, so we’re using that at home, and it’s kind of transparent to us, but, you know, there will be a time when I’m trying to go to a website or something and it’s going to block the traffic, and kind of deal with that.
You mentioned public WiFi, do you recommend that most people have some sort of a VPN is they’re going to do any … Or just not do any sort of, like, serious browsing if you’re on public WiFi, if you’re going to log in to, like, something sensitive, like-
Aaron Campbell: Yeah.
MikeDemo: … your bank.
Aaron Campbell: So, this touches on, really, kind of the third thing that I also point out to people as well. So, VPNs are getting easier, but I still think the barrier to entry for using them tends to be more complex than your average user wants to deal with, and I get that. So, generally speaking, I would say there are some things you probably don’t want to do on public WiFi, but there’s a lot of things that are probably fine to do on public WiFi without a VPN.
Having said that, every website ever should have SSL, and this helps a lot around public WiFi, right?
Aaron Campbell: In the point to point encryption that encrypts from the computer all the way to the server, or, you know, along the whole route, so that anything, sort of, packet sniffing, anything that’s looking at the traffic between your laptop at the coffee shop and your server that has the website on it, is getting encrypted traffic that they then have to try to break the encryption on, Is hugely helpful.
And, so, not that SSL is free many places, dirt cheap any place that it’s not free-
MikeDemo: Yeah, through, like, projects like Lets Encrypt [crosstalk 00:24:44]
Aaron Campbell: Let’s Encrypt, even, sometimes, people find it easier, if their host doesn’t offer Let’s Encrypt, to just pay $5 bucks to get it from their host because it auto installs or whatever it is, but yeah, it’s so inexpensive, and it’s another one of those big things that you can do, and I tell everyone that that belongs on every site, from my parents’ blog all the way to the bank website. Like, every site should have SSL at this point. That will help.
MikeDemo: Do you think the push for SSL has had a side consequence? Because when I talk to users, and they’re like, well, how did my site get hacked, I have an SSL certificate? So, do you think this huge push for SSL is going a disservice with confusing what it is and what it isn’t? Because SSL is important, but it serves a very different purpose than other security products that might be out there such as a web application firewall. You know, an SSL isn’t going to make your site hack proof-
Aaron Campbell: Right.
MikeDemo: … in and of itself.
Aaron Campbell: Yeah, there’s no one thing, especially not one thing that’s free or $5 Dollars a year that’s going to make your site hack proof. That’s not very realistic. I think that SSL, like so many other technologies, as it breaks from sort of the technosphere, this group of tech people that really understand it, into the general public, where you … I do expect, you know, that someone … Like, my parents would obviously just ask me to fix whatever it is, but someone that’s of my parents’ capabilities is now kind of expected to install SSL on their own. Like, that’s a thing that anyone can do for their website. And, as it makes that transition, there’s going to be a lot of misunderstandings, just like there is on any technology.
I’m kind of okay with it, because SSL really is a fantastic push toward both security and privacy, which I think are both important. So, there will be some misunderstandings. For everybody that’s watching or listening, SSL is one thing that you should do, but it’s not the only thing you should do. SSL doesn’t mean that you can have your password as password. You kind of need to do all these things, but … Yeah.
MikeDemo: Well, one last question, because we kind of ran out of time, and we have to go get pizza.
Aaron Campbell: Yes!
MikeDemo: So, that’s exciting.
Aaron Campbell: Pizza. I’ve been promised coal-fired pizza-
MikeDemo: Coal-fired pizza.
Aaron Campbell: … which I’ve never had before, so-
MikeDemo: Yeah, let’s bring him to a Minnesota place. We always ask, what is your favorite WordPress or WordCamp story?
Aaron Campbell: WordPress or WordCamp story, huh? So, this is a tough one. I’ve been in WordPress for a long time, so I had a tough time trying to sift through and figure out one thing, you know? I think that you touched, a little bit ago, on sort of how being a part of an open source project is often more than just the software, it’s the other people that you meet, the kind of friendships, even, that get formed there, so, that’s definitely kind of the best thing that’s happened as being a part of WordPress.
But, when it comes to WordCamps, I have to say that the coolest thing is that I’ve been lucky enough to get to travel around to camps, and get to find quirky little things at each place that I go. And my favorite thing recently, within the last year, was in Baltimore.
I went with a group of people that said, “We’re going to show you something just unique about Baltimore,” and they took me to Mr. Trash Wheel, and all the Baltimore people are going to be laughing right now, but Mr. Trash Wheel, which is this thing that they have for cleaning up their waterways. It eats trash, and they’ve put giant googly eyes on it, and it has its own Twitter account, and … I don’t know, like, there are some things that are so quirky and unique that I would never have gotten to see if I hadn’t gone to a WordCamp there. But Mr. Trash Wheel cleans up trash from the waterway and, like, tweets on how much trash it’s eaten, and it was hilarious, and I think about it all the time, so …
MikeDemo: Yeah, and then at, heck, WordCamp Phoenix, just last fall, we ended up going to a Skyline Display Dealership to get a booth fixed, and then we had Mexican food for dinner, so crazy-
Aaron Campbell: Yes. Yeah.
MikeDemo: … little things happen, so it’s interesting.
Aaron Campbell: All sorts of crazy adventures, yeah, we got a booth patched, that’s true.
MikeDemo: Saturday night.
Aaron Campbell: Yeah.
MikeDemo: And then you gave me-
Aaron Campbell: All kinds of fun.
MikeDemo: … a tour of the GoDaddy office-
Aaron Campbell: Yes.
MikeDemo: … which was exciting.
Aaron Campbell: Yeah.
MikeDemo: We did not do the slides though, so …
Aaron Campbell: We didn’t do the slide. Next time.
MikeDemo: Yeah, I didn’t the waiver. They were like, “Do you want to sign a waiver?” And I’m like, “What? For what?”
Aaron Campbell: Then we were at the top of the slide, and you wished have-
MikeDemo: Signed the waiver-
Aaron Campbell: Yeah.
MikeDemo: … exactly. Well, next time.
Aaron Campbell: Next time we’ll know.
MikeDemo: Next time. Well, thank you so much. I definitely appreciate your time.
Aaron Campbell: Absolutely [crosstalk 00:30:09]
MikeDemo: How can people follow your online?
Aaron Campbell: You can check out my website at AaronDCampbell.com, and on most of the social networks, like Twitter, I’m just @AaronCampbell.
MikeDemo: Yep, and it’s also at the bottom of your screen, you can see his-
Aaron Campbell: Oh, there it is. Look at that [crosstalk 00:30:24]
MikeDemo: … Twitter account, right there, right there.
Aaron Campbell: Twitter, fantastic.
MikeDemo: You can also go to BoldGrid.com, and the show notes in there, and we’ll have a transcription up in about 24 hours, and we’re linking to Aaron’s GitHub account, his WordPress profile account, and his Twitter account as well.
Aaron Campbell: And I think maybe we can also throw in a link to where to report WordPress security issues, because I like to throw that out for everybody.
MikeDemo: We definitely can do that.
Aaron Campbell: Yes. So, we’ll throw that in there as well, and that will be at HackerOne.com/WordPress.
MikeDemo: Excellent, and we’ll put a link to that in the show notes.
Aaron Campbell: Awesome.
MikeDemo: Thank you so much, again.
Aaron Campbell: Thanks.
MikeDemo: Everyone have a good night.