Eden Full Goh: Filip, it would be awesome if you could just share a little bit about how you started working in tech and how you got a chance to move to Silicon Valley.

Filip Grebowski: Yeah, sure. So I studied Computer Science at the University of Kent in the UK, and the degree was actually Computer Science with Artificial Intelligence, although I've nothing to do with AI right now other than ChatGPT-4 and all that hype that's been going around. Essentially, as part of my university degree there was an opportunity to take a sandwich year. What that means is that we can fly out for a year to do our internship at some other company.

Now, usually most companies are based actually in the UK but I was one of the lucky 40 to be selected to join Cisco in Silicon Valley to work as a software engineer, specifically in a team that was responsible for automation testing of the internal systems. It was called the DNAC, the Digital Network Architecture Center, and I was one of those people on the team that actually wrote those selenium tests and wrote that automation to test the website to make sure everything works as it is.

Eden: That's awesome. I guess out of curiosity, when you were studying in university to become a software engineer, I'm sure there's your algorithms and data structures classes. There's a lot of best practices about operating systems and architecture and design. But in terms of jumping into testing and getting into that industry as a fresh out of university student, how do you learn about testing best practices? Learning about selenium and how to get started there?

Filip: So I think one thing you start to realize once you actually start working in industry, and I feel like a lot of people feel the same way, as that you're put to certain tasks that either they never taught you, either you never heard of, or it's just something completely new that you're expected to learn.

I think there is nothing better than learning while in the industry because there is that healthy pressure that they put on you to actually learn new things.

Automation testing with selenium was definitely one of those things for me. It was something completely new, I had absolutely no idea how to do it. But luckily enough, when you're on an internship or whenever you join a new job, they're fairly aware that they don't expect you to know everything, and they give you the tools and the resources and the help you need to actually learn it.

So it was a little bit of a learning curve for me, for sure. It took me a while to understand why it's necessary, why testing is so important in certain cases, and how to do it properly because of course the one thing is knowing how to write tests. The other thing is writing tests that actually work and are performant, and check for the right solution because you can also make mistakes in writing tests that can then become faulty and give you an inaccurate result of what you might expect.

Eden: I see. So I saw on your profile that you were at Cisco for four years in total, and so for half of that time you were building and leading the UI/UX development for the test ops team. What is the test ops team at Cisco? And is this an internal team? And how many products does that span?

Filip: So, yes, that's true. I actually ended up leading the developer efforts of all the internal dashboards. Now, these were internal tools and there were many. I actually don't know how many in total. I was responsible for managing three of them, there were much more than that because Cisco has very, very many different sectors and different internal teams that manage a lot of different products.

But of course, yes, three of them were quite important and they were related to the main product which I mentioned, which was DNAC. The way we managed it, it was to make sure that the quality of all the tests, all those dashboards were designed to track all the tests that were written throughout the whole application for the whole system. This wasn't just selenium tests on the web UI that we actually run, but it was also tests to do with other parts of that product.

All of that information was accumulated into one space, and I was the person responsible for making sure that it's displayed, making sure that it's easily accessible and easy to read, of course, so it's understandable.

Eden: This is interesting, because I spend a lot of time thinking about testing, regression testing, analytics and a lot of our customers at Mobot are trying to better understand the health of the product. After you complete a batch of tests, how do you know something is ready to ship, that nothing is wrong with it? You feel like you have the peace of mind.

It's interesting to know that you spent quite a bit of time refining and streamlining and optimizing a lot of tooling internally, I would presume to just give your team and all of the other Cisco engineering teams peace of mind that things are going well. In your experience, what are some metrics that are good to present as part of a dashboard metrics that are important to actually gauge the health of a product and readiness to release?

And what are some metrics that are maybe kind of a vanity metric that you show on a dashboard, but it's not actually useful?

‍

Parsing Useful Metrics From Vanity Metrics
‍

Filip: Right. I think that's a really interesting question actually, because this was part of a bigger internal debate of what we actually show. Now what you have to realize that there is a lot of parts of the team that work on this and their information has to be relevant to different sectors of a team. There is the really concrete engineers who just like seeing the numbers and the data, but these dashboards also have to appeal to the managers and the higher up people, the corporate stuff who just want to see the fundamentals and have insight into how the product is going and if it's functioning properly, and if the data is up to par with their expectations.

Now, of course there is a lot of dependencies here that come with a specific product and what might be necessary to show for this or for another different product, but the general insight is that you always want to have the overall summary of what's going on. This is important for everyone on the team, this provides information for everyone that's relatable to see the progress.

I think that there is a few ideas of why dashboards are also necessary. One of the ideas, just like I've mentioned, to make sure that we can track the progress, but two, actually dashboards also serve a very important purpose of motivating in some way, parts of the team members to say that, "Yes, we wrote these tests and 95% of them are passing which means that we did a good job and we can continue and move onto something else."

So the different parts, it's very hard to specify what exactly you should show because it's very product dependent.

But in general, keeping those two things in mind, that the overall information, the most important information about a product should be somewhat summarized if there is numbers involved, and also if you're working, for example, on a critical feature that is also quite relevant, then any kind of data in regards to that feature should also be displayed.

Eden: Interesting. Yeah, because one of the challenges that I've sometimes encountered in working with engineers is over time as you build a product, you create a lot of test cases and then you just keep adding them in, you keep adding them to the dashboard and the bottom denominator of number of total test cases goes from 300 to 400, to 500, to 600.

What I've often found or observed is that engineers aren't always curating or thinking, "Is this test case still relevant? Is it actually useful?" And over time if you keep accumulating test cases and you have this 90% pass rate of all your test cases, 95% pass rate, but let's say half of those test cases or a quarter of those test cases are not actually really testing anything or it's giving you a false pass almost because it's going to be almost impossible for that test to fail.

What's your perspective on that? What would you recommend for engineers or other test ops team members who are trying to keep a dashboard healthy, keep a process healthy? How do you make sure that things don't go stale?

Filip: Yeah. So tests in general, they can be perceived quite differently. Obviously as time goes by you accumulate different test cases and scenarios might arise where maybe later on, let's say a year later, you're testing for something and suddenly those older test cases don't work any more and they start to fail. Then suddenly your score goes down because of test cases that might not be relevant anymore.

This is actually quite interesting because what we do at my company, at PermitIO right now, is also something similar where we don't necessarily deal with tests but we deal with creating policies and authorization. But the same rule applies that there is certain policies that users might want to test, that were previously created but might not work anymore, and there is these feature replays that we're trying to implement.

The same would apply for tests, there should be some sort of feature system built into testing where you can replay the previous tests from a specific section of time and then see how they perform and see if they're relevant, and see if they are to par with what your expectations are. That will actually make it much easier to segregate, to maybe remove or concretely accumulate the tests and maybe in this case, maybe the policies from our side that are not necessary. It makes the management of that previous archived data much, much nicer.

Eden: Let's switch gears and talk a little bit about your current role as a developer advocate, and also some of the work that you do on YouTube where you're teaching students all over the world about web development. That's a topic that's very near and dear to me, 10 years ago I took a programming bootcamp, I also learned a lot just from taking online courses and tutorials and everything. It's awesome to know that in 2023, just the way that software engineering education is approached is so different. I'd love to hear a bit more about your decision to transition into a developer advocate and content creator role.

Developer Advocacy and Content Creation

Filip: Right, yes. I think it's quite similar to maybe what many would say and I always had a thing for videography and filming. As I started to become much more passionate about someway of coding, whether it is just frontend development to any part that actually brought me some interest, I decided that I would combine the two.

I would try and work with the two to make something cool, so that's where the idea came from where I essentially was able to create content that raps around the idea of coding and make it creative, fun and appealing to the audience. I think coding in general, sometimes it can be a little bit intimidating for the ones that are starting at the very beginning, and shining a different light and a different perspective on actually how creative and fun it is, and how much possibilities you have.

It really encourages others to maybe start to do something, to move into that direction. That's where it all started to flourish, where I started making those videos and things started to pick up. Actually the first video I did was the React Native versus Flutter that somewhat really actually started making my channel grow. It wasn't ever anything that I expected to happen, and obviously with video creation, with content, with speaking to a larger audience, that's when developer advocacy started to come in.

It's a fairly new role, now it's actually becoming much, much more popular in companies because, yes, we are appealing to much more of a digital audience, yes, we need people that are also technical but speakers that can go and present and go and talk about a product, and show the benefits of it. So that transition almost seemed like it's natural, like it's very favorable to my job position, and I decided to test it out. Right now I'm absolutely in love with it.

Eden: It's interesting that you're saying one of your most successful videos is about React Native versus Flutter for cross platform mobile apps. I'm curious why that particular video took off as much as it did. Do you feel like there's not enough content out there about mobile specific technologies? Or what do you think really resonated with your audience about that?

Filip: I think at that point there was always a debate, and it was which platform, which kind of tool do I use to develop these applications? There was always answers that said you should use React Native, and there are always answers that you should use Flutter. But there weren't any concrete reasons why you should do so.

There was always a 50:50 balance and someone who came to watch or read an article about it came out not really having an answer, and I decided that I would approach it in a different way and I would actually give an answer and showcase scenarios where one may be more beneficial than the other. I guess it was fairly interesting, people actually resonated with it, and so that video started to spark more attention than I expected. Like I said, it was one of those first videos that brought eyes to my channel and brought eyes to the things that I do.

Eden: I guess to recap a little bit for any of our audience members who haven't seen that video, would you be able to give a little summary of when is it good to use React Native and when is it good to use Flutter? I'm curious, since you've released that video to today, has that perspective changed at all as you've learned more about these technologies?

Filip: Right. I think actually one of the main reasons here or the differences that was quite interesting with React Native and Flutter at that point was the type of emulators that you get and how you can actually test and visualize your apps. Of course the bigger appealing thing there was that Flutter required you to learn a new language, to learn a new way of writing those apps, whereas React Native was quite familiar to the people who already worked with React.

So from what I remember, one of the concrete things I said is that if you have experience with React and if you know how to use it, you should definitely use React Native because why would you go ahead to learn something new? Yes, there are cases where Flutter might be more performant. Of course now things have changed and I'm not actually fully aware what the situation looks like right now, but maybe things changed and maybe now Flutter is more recommended for people who are learning React or maybe not React.

But at that moment in time it was a fairly concrete conclusion. There isn't time for you to maybe investigate. I think one of the things developers most often miss, and I think it's becoming really popular right now is that if there are tools out there that already allow you to do something maybe that you don't have experience with or maybe that give you that block of moving forward, then you should always go with a solution that's available, that's easy to implement so you can actually focus on the value of your product and the value of something that you're building.

There is often times where you devote time of your developers on the team to build something, which you could already use with a different tool that's available out there. So why would you waste those developer resources on doing something where you don't have to do it? So it's the question of do you have to build something or maybe you don't, and you can utilize something that's already out there and move forward with whatever you're implementing.

Eden: Gotcha. There's a lot that's changing in this industry, every few months I feel like there's a new library, there's something new, a new framework that's come out, a new SDK. Engineers are constantly having to be agile and responsive and keep up to date with the latest technologies and frameworks, it can sometimes be overwhelming. What are your recommendations for engineers and DevOps, testing folks that are looking to learn always about the latest and greatest?

And how do you make decisions about whether or not to, "Okay, I used to use React Native and now I'm going to use Flutter"? Who knows, in another two months, something new is going to come out. What are some of the considerations that folks should be thinking through?

Keeping Up with the Latest Technology

Filip: Right. I think in general it's quite a hard question to answer and I'll tell you why. It's because there is so many things that are coming out recently, I want to say it's almost impossible... I'll say it is impossible to follow along with everything, and I think sometimes we need to essentially realize that. I know there is a lot of anxiety coming from the side of developers where they feel like they need to know everything but they find it so difficult because it is difficult. It's impossible. That's why we start to niche down into what we actually enjoy and then follow those specific topics.

The advice I would give is if you are interested in a particular part of tech, whatever it might be, whether it is frontend develop, whether it is maybe even more niche and creative web development, things like Free JS and WebGL and all sorts of libraries like that, whether you're into backend or you're into cloud or whatever it might be that you're interested in. If you have that particular focus, you should be starting to specialize in that focus.

Of course we always have to be aware that there are overall technologies that are upcoming and somewhat we should have that awareness that they exist, and maybe what they do. But we don't have to always learn them and know everything. Nobody knows everything, and just having a specialty in one subject is much more than good enough for the specific kind of tech job positions that you might be looking for, if that's the case.

Eden: Yeah. So I guess on your YouTube channel, you are teaching a lot of engineering students web development. I'm assuming some of them are starting from scratch, this is maybe their first exposure to software development. Sometimes I worry that everyone's learning the same tutorials, the same web development best practices. But there's not enough of a focus as well on teaching early on DevOps or testing or some of these other best practices which are not only just learning the tools, learning to code, but how do you actually be an effective, productive software engineer or a test engineer? How do you strike that balance with your students?

Filip: So of course it's quite hard to, for example, throw everything at a novice audience, right? You don't want to scare them too much, and I think speaking about DevOps or SecOps or maybe just implementing any type of security in a wrap, it's quite hard. So the way to approach it is to realize that there are several building blocks to any application and there are certain building blocks of any team, for example.

A very simple example is previously as we were developing, let's maybe go back five or six years, the main thing we would focus on is how to build a part of a website and how to give it some logic, maybe with some backend Node JS or something similar. Now because the world has evolved and security is much in a higher place, we have to focus not only just a basic app, but also on authenticating users, how do you prove their identity?

On authorizing users, how do you make sure they have the right roles to perform certain actions? You focus on writing specific tests for everything they do within the app to make sure that they don't do something wrong or to make sure that everything is functioning properly. Then you have the whole infrastructure, how do you make sure you scale the app? How do you make sure that you grow and it can actually be performant and it can be fast with lower latency?

So it's all part of the bigger building blocks. I think for any kind of new person that maybe is willing to learn, you have to always start small and, yes, of course it does get repetitive. It often overlaps, people teaching the same thing, but at one point especially with me and I know I had a lot of people message me about this as well, it's that at one point something clicks. I think we can all relate to that as developers.

At one point you learn something and it clicks, and you start to engage in the learning more yourself. You start to be more interested in something and you read an extra article and you read another one and another one, watch a different video. Somewhat you really start to shape what you're interested in and you follow in that direction. I think one mistake we are making though, like you've mentioned, is that we're not letting the novice people know enough that there's many building blocks of a great application.

Also that they are sometimes even more important than the actual application itself because they might cause bigger problems in the long run as you're developing it. For example, not having enough security embedded into your application right. So I think just making people aware that many different building blocks do exist is very, very important, and I can see already different creators implement that approach and make people aware.

Eden: One of the things that I find really interesting about your role as a content creator and a developer advocate, is your audience is technical and generally in my experience, at least when I've worked with a technical audience, people often have very strong opinions, ideas, about what they want to use or the right way that something should be built.

Whether it's programming the best programing language, the best tech stack, test driven development or what framework to use. When you're working as a developer advocate and a content creator, how do you make sure that you provide a balanced perspective, but that you're also able to change people's minds? How hard is it to convince someone to use a completely different tool and try a different practice?

I feel like that's why this role is so necessary, is it's not just about engineering, it's about awareness and education and empowerment at all levels, whether you're a senior engineer or a chief architect or a new web developer. Yeah, how do you build your position to change people's minds?

Reaching an Audience and Changing Minds

Filip: Really, really good question. It really depends who you are speaking to and who is your audience. For a fact I know that developers hate being advertised to. That's one of the things you really have to avoid doing when you're trying to pitch something new to someone. Now, it's definitely going to be much harder to convince someone who's really experienced into using a different tool because they might just be so accustomed to the things that they do, and that's purely because they're comfortable with doing so.

Why would they need to learn something new? It's a barrier that's very difficult to cross. It's a different story with people who are maybe more novice because they're more open to learning new things and they're more open to listening to advice, as such. So the audience here plays a really important role as to how you want to portray the information you want to sell to them. I think with any developer, whether they're a novice or very experienced, the main convincing factor is to give them concrete examples of why it might be necessary.

I can give you an example from our company, PermitIO. Like I said, we do full stack authorization and one of the things we are doing is tackling one of those building blocks, which is authorization. So how do you convince all of these developers who are already in these big companies who build their own home brew solution to suddenly delegate to a different product?

Well, that's extremely hard because if you tell them that they should do it, they're not going to listen to you. It doesn't work like that. You need to come at them at a different angle, you need to make them aware that there is vulnerabilities in doing so themselves, for example. You need to build that awareness and show them examples of why it might be unnecessary, why it might be beneficial to do it a different way with a different solution.

With that kind of approach you start to open up their eyes because they're more eager to listen. You're not just telling them to do so with advertising, but you're showing them the things that could go wrong, the approaches that might be unnecessary, and giving them examples of approaching it in a different way. That's the best PLG aspect, Product Led Growth aspect, of targeting any developer whether they're someone novice or someone experienced to rethink maybe their current solution and approach it at a different angle.

Eden: Yeah. That really resonates with me because I think we've often seen that with the way that we're building Mobot, it is an automated testing solution that uses mechanical robots to do mobile testing, and when people hear about this their first reaction is always, "Really? A real mechanical robot? Is that a joke? I've never heard of this before. This is so different."

It's so unintuitive to some people that, yeah, we have to go through this exercise and we have to take the time to, like you said, show examples, showing you when are there issues that bugs, defects, that can't be remediated by testing in a simulator. This goes back to what you were saying earlier about React Native versus Flutter, you might get a different experience when you build an SDK into a React Native app versus a Flutter app.

And so I think those are some of the things that calling out specific examples and just giving them the tools, the stories, the anecdotes, the case studies. Kind of like you were saying with PLG, engineers want to be able to try something out for themselves, and I can totally see this is why that role of developer advocate is so important in 2023. You have to be able to give people the opportunity to experience your methodology, it's not just a product anymore, there's also the methodology and best practices behind that.

Filip: Of course. I think with what you guys do at Mobot, it's actually quite relevant and there is a lot of educating to do there because people nowadays think, "Okay, I can just run something in an emulator and suddenly it's going to work, and it's going to perform the way as it would when someone is actually using it in a physical way." But that's really not true, because there is a lot of things that you can't achieve from an emulator that maybe you can achieve from an actual physical machine.

For example, you have features like haptic feedback in devices and you maybe need to think about how you're holding the phone to adjust what appears on the website to have good accessibility. That's not something that can necessarily be tested on an emulator on a virtual device on the screen. Yes, it covers maybe a few relevant things, but it definitely doesn't cover all of them.

So having something that's physical and having something that imitates a real user is definitely advantageous and I can see that being used in many companies and actually benefiting many companies to make sure that something that they do release actually makes sense because a lot of the time... I've even seen a few examples of people, for example, building apps, different companies building apps and releasing something and suddenly when I start using it as to what they show, it's a completely different experience. It just proves to me that it hasn't been thoroughly tested, something was missing there.

Eden: Yeah, it's interesting because I think this is a problem that's going to get more complex in the next five to ten years. I think it applies broadly to not only mobile but also software engineering in general, there is such a fragmentation of methodologies, programing languages, tech stacks, SDKs, libraries and tools.

I also think there's incredible opportunity that's associated with all of this, but it does mean that the types of services and software that everyone, also products, that we all need to support... We're creating more responsibility on to individual engineers because now they need to be thinking about all the use cases, thinking about all the test cases.

Filip: Yeah, I was just going to say. It's not just that because with the way we develop now, we want things to be as fast as possible and we want most things to be somewhat automated. Now, I cannot imagine building an app and having to, for example, test it on an emulator on all different devices. It just won't reflect the actual performance. Then again, buying 100 devices, it doesn't seem like a great business thing to do, right? That's, for example, where your company comes in and they do a wonderful job about that.

Eden: Yeah. It's exciting. I think there's more opportunity with more different hardware devices, more platforms, more operating systems. It feels like every 10 years the technology ecosystem really does change. And so looking to the future, my last question for you is how are you thinking about the future of developer advocacy and education for new members of the industry that are new and coming in? How do you think that's going to change in the next three to five years as tools like GPT really change the way that we use and think about learning and day to day tooling? How do you think this is going to evolve in the next few years?

AI and the Future of Developer Advocacy

Filip: I think it's definitely going to be somewhat of a revolution. I think we're looking to really expand the way we look at engineering roles and how they work, and what is expected of them. But, for example, targeting that AI kind of stance and if that's going to play an impact on the way we perform as engineers, yes, absolutely. I think there is a really big dilemma and I've got a million questions about it already, like will it affect our jobs? Will it kill my six years of learning that I did?

And the answer to that is no, it will not. It's about adjusting and learning to use and work with the new technology to benefit us as engineers, and not necessarily about it taking our jobs and making us not perform the way we were expected or make our learning go to waste. Your learning never goes to waste, it's always very necessary and important. Now, when it comes to the developer advocacy side, we can already see a huge involvement in that.

Companies are starting to hire these people that are the voice of the company, that portray and teach the world about how to use them and why it's important. So we're definitely going to see more of that, we're going to see more companies adopt that kind of strategy.

I think we're going to see more and more companies become PLG because in the end the best way to have a great community and an audience in any company is to have developers who are fascinated about the product using your product because in that way you get the best organic growth. So more developer advocates, more different stage appearances are going to come, and I think the talks will become much more interesting.

I think there is a huge shift in developer advocacy to actually not just speaking, but showing how things work and impressing people by how maybe the company that you're working for is selling and is providing a solution to a feature. I think we're definitely going to see more of that as well.

Eden: Yeah, it'll be really interesting to see how the next few years turn out. I've already, in the last month, seen all of these different tutorials on how to use GPT to learn programing, learn software engineering, and I can only imagine that if we have the opportunity to integrate these kinds of tools into every single product, it's going to... I think not only for just learning, but for everyday DevOps, for infrastructure and tooling, and testing in particular.

I'm really excited about how this can impact test case design, test case management, exploratory testing, integration with UI level and interpreting a UI to have a more human like testing experience. There's a lot of cool stuff that's going on and I think definitely a very exciting time to be at the forefront of engineering.

Filip: Absolutely. There is already a large amount of companies using AI in their applications. I can give you two examples. For example, Duolingo is using AI to make sure that the conversation that the user is having in another language is much more coherent and much more easy to follow along with. You have companies like Stripe using it for fraud detection.

So I think different companies are going to definitely start to adopt this, and I'm intrigued specifically from a developer advocate perspective how AI is going to be adopted into documentation because I think there is a lot of very, very interesting things that can be done there. But I'll leave it at that.

Eden: Yeah. Oh man, we could probably spend a whole other podcast episode just talking about ways to use ChatGPT for documentation and learning and everything. We'll have to save that for another episode. Thank you, Filip, for joining me on the podcast. I really enjoyed our conversation today. I think you touched on a lot of different aspects of software engineering.

I think it's not just about web development, right? There's advocacy, there's education, there's infrastructure, there's security, there's testing, there's so many different parts of being a good software engineer, and I think it's awesome to know that you have an audience and your role is advocating for these kind of best practices. It's really interesting and inspiring.

Filip: I 100% agree. Thank you for having me on the podcast. It was an absolute pleasure to be here and to discuss these interesting topics.