Home GADGETS Debugging UI with AI: GitHub Copilot agent mode meets MCP servers

Debugging UI with AI: GitHub Copilot agent mode meets MCP servers

Debugging UI with AI: GitHub Copilot agent mode meets MCP servers

If you’ve ever dusted off an old project and thought, “How did I leave things in such a mess?”, you’re in good company.

On my latest Rubber Duck Thursdays stream, I dove back into my OctoArcade Next.js app, only to rediscover a host of UI gremlins. So, we experimented with something that felt like magic: letting GitHub Copilot agent mode, paired with Playwright MCP server, identify and fix UI bugs. Along the way, I learned (again) how crucial it is to provide AI tools like Copilot with clear, detailed requirements. 

Let’s walk through how I used these agentic tools to debug, test, and (mostly) solve some tricky layout issues, while covering practical tips for anyone looking to leverage Copilot’s agent workflows in real-world projects.

The setup: Revisiting OctoArcade (and its bugs)

I started by firing up OctoArcade, my collection of GitHub-themed mini-games built with Next.js and TypeScript. Within minutes, I realized I had been introducing a new game to the app, but hadn’t quite gotten around to fixing some bugs.

Here’s what we accomplished in one stream session:

  • Problem: Navigation header overlapping game content across all games
  • Solution: Copilot agent mode and Playwright MCP server identified the issue through visual inspection, and implemented a global header fix
  • Bonus: Fixed some additional UI issues (unintended gaps between the game canvas and footer) discovered during testing
  • Result: Hands-off debugging that solved problems I’d stepped away from, and had previously spent some cycles on fixing

Let me walk you through how this worked and what you can learn for your own debugging workflows.

Making sure Copilot custom instructions are set up

With my environment set up in VS Code Insiders, I checked that my Copilot custom instruction files (.github/copilot-instructions.md, *.instructions.md files) were up to date. This is usually my first step before using any agentic features, as these instructions provide important context on my expectations, coding styles, and working practices — influencing how Copilot responds and interacts with my codebase.

In my last blog post, we spent time exploring recommended practices when setting up Copilot custom instructions. We also covered how the copilot-setup-steps.yml sets up a developer environment when using Copilot coding agent. Take a look at that blog post on using GitHub Copilot coding agents to refactor and automate developer workflows to learn more.

Always keep your Copilot custom instructions current (including descriptions of your repository structure, common steps like building and testing, and any expectations before making commits). Copilot agents depend on this context to deliver relevant changes. When I think my instructions file is out of date, I typically prompt Copilot in agent mode with a prompt along the lines of:

Based on the #codebase, please can you update the custom instructions file for accuracy? Please make sure to keep the structure (i.e. headings etc.) as-is. Thanks!

In some of my instruction files, I’ve even instructed Copilot to keep key documentation (README, .github/copilot-instructions.md, etc.) up to date when it makes significant changes (like refactoring files or adding new features).

Agentic debugging: UI troubleshooting with Playwright MCP

Playwright MCP server is a powerful tool for end-to-end testing and UI automation. Since it’s an MCP server, you can access it through your favorite AI tools that support the Model Context Protocol, like Copilot agent mode and Copilot coding agent! In agent mode, Copilot can use Playwright’s structured tools to:

  • Load web pages
  • Simulate user actions (clicks, navigation)
  • Inspect rendered layouts without needing vision models

This means you can ask Copilot to “see” what a human would, spot layout issues, and even propose CSS or component fixes. To get started with Playwright, it’s as easy as adding the below to your MCP configuration:

{

  "mcpServers": {

    "playwright": {

      "command": "npx",

      "args": ["@playwright/mcp@latest"]

    }

  }

}

Once you have started the MCP server, you should see that Copilot now has access to a suite of new tools for browser interaction like:

  • browser_snapshot – Capture accessibility snapshots of pages
  • browser_navigate – Navigate to URLs
  • browser_click, browser_type, browser_hover – Interact with elements
  • browser_resize – Test different viewport sizes
  • browser_take_screenshot – Visual documentation
  • And many more: You can find the full list in the tools section of Playwright MCP server’s README

With access to a new set of tools to solve the UI challenges, it was time to point Copilot at the problem. Meaning, I now had the task of clearly defining the requirements in my initial prompt…easier said than done.

The debugging journey: Real-time fixes and lessons learned

1. Describe the problem and let agent mode work

I noticed that, in several pages, the main content was tucked behind the navigation bar. This was particularly noticeable on any pages that rendered games. On some pages (like OctoPong), I saw inconsistent spacing between game elements and the footer.

To get Copilot agent mode started, I aimed to be as explicit as possible in my prompts:

I have spotted that there is a bit of a UI error. It seems like the main content of any page "starts" behind the navigation bar. This is more evident on the games like octosnap, octopong and octobrickbreaker.

Can you take a look at the site using Playwright (you'll need to spin up an instance of the server), take a look at the pages, and then investigate? Thanks!

It loaded up the pages to configure each game, but didn’t try loading the games themselves (so missed some context). I followed up in a separate prompt:

Sorry, I wanted you to take a look when a game is actually loaded too. Can you play the game Octopong and Octosnap – I think it’s very visible in those? Do that before you build a plan.

Lesson: The more context and specifics you provide, the better Copilot performs, just like a teammate. 

After spinning up Playwright MCP, I watched Copilot:

  • Launch a browser
  • Navigate through the app’s pages
  • Diagnose where and why content was hidden or misaligned

Ultimately, we had to evolve the way that we were rendering the navigation bar. The current implementation had a separate navigation bar (DynamicHeader component) on each of the game pages with its own local state, overlaying the “main” navigation bar. Instead, Copilot suggested using the navigation bar from the root layout and passing the relevant context up, so that only one component is updated and the root layout gets updated as needed..

Hands-off debugging: At this stage, I was literally hands off, watching as Copilot tried fixes, reran the app, and checked the results visually. As it implemented the new approach with a new header-context file, Copilot recognized linting errors, and iteratively fixed them.

2. Iterating on UI requirements

Fixing bugs is rarely one-and-done. I noticed another bug, specifically for OctoPong. There was a small gap between the game board and the footer, which didn’t show up clearly on the livestream, but was noticeable on my own screen. Fortunately as developers, we’re used to small iterative and incremental improvements.

So once again, I turned to Copilot. However, as I iterated, Copilot would make changes, but they didn’t fully achieve what I needed. The problem wasn’t Copilot though; it was me and my unclear requirements.

Prompt Result Reflection
I’ve noticed a minor UI bug on the Octopong game page (this only happens when the game is actually live). There is a small space between the game itself and the footer. I want the game to extend all the way to the footer (not necessarily push the footer beyond the fold though). Can you use the Playwright MCP server to explore what’s going on, build a structured plan / todo list to resolve the actions? Thanks It achieved what I had asked, but the pong paddles no longer displayed (which was a side effect of the container now being 0 height). Through no fault of Copilot, I hadn’t asked for the game components (e.g. Paddle/Ball) to be visible in the game area. The game was still playing (in a 0 height container), but the key components were not visible to me as the player. – Good clarity on tools to use.

– Good clarity on asking for a plan (as Copilot asked me to review/approve before making the changes).

– Lack of clarity on the full requirements (i.e. having the game components be visible and working).

Just to jump in, can you test again? It looks like the paddles and the ball are now missing as a result of the change? The game began working again, however it introduced the gap that we early sought to resolve. – Solved the immediate challenge of making game components visible.

– Lack of clarity that the earlier requirements were still required.

Can you check the game once again? The spacing issues are still there.

The requirements are:

1. The game is playable (i.e. balls and paddles are visible and one paddle is usable for the player).
2. The game area covers the “full space” between header and footer.

I think the space problem is back. You must meet both requirements please, thanks.

Once again, Copilot fulfilled the requirements! But this time, the game extended beyond the viewport, and so a user would have to scroll to move the paddle to prevent the ball scoring against them (which is not an ideal experience!). – Solved all of the requirements we outlined

– Lack of clarity in the actual requirements (that the game should not extend beyond the viewport).

Thanks! Sorry, I forgot to give you a third requirement. Your solution makes the game extend beyond the fold,  which makes the user have to “scroll” to play the game.

These are the requirements you must meet:

1. The game is fully functional (paddles/ball working and visible).
2. There is no space between the game and the footer.
3. The game must not extend beyond the fold (i.e. the user must not have to scroll to see any part of the game board. So the game board at maximum must end at the bottom of the screen). The footer can be below the fold.

Please feel free to reword/rewrite my requirements, as I struggled to define  them. Make sure you confirm with me the requirements are accurate.

Success! After we prompted Copilot with our full requirements, it was able to think through and iteratively approach the problem to get to the working layout. – A full set of requirements solved all of the requirements we outlined.

– While there were some minor issues in the mobile view, (a small gap between the navigation bar and game), other pages hadn’t yet been optimised for mobile. Since this isn’t a priority, it can be a task for later. 

Each prompt brought incremental improvements, but also new side effects (like games extending past the viewport or missing paddles in Pong).

Key insight: Context is important, and making sure that we clearly articulate our requirements is a key part of that. The biggest challenge wasn’t technical, but precisely describing what I wanted. Getting the requirements right took several attempts…And lots of feedback from our viewers in the livestream chat. 

Practical tips: Working with Copilot agent mode and MCP servers

Here’s what I learned (or relearned) during this session:

  • Keep Copilot custom instructions up to date: The agent relies on these files for repo context and best practices.
  • Give Copilot more power with MCP: Playwright MCP enables true end-to-end testing and UI inspection, making it invaluable for debugging complex web apps.
  • Be explicit with requirements: Like any collaborator, Copilot only knows what you tell it. List out your must-haves, expected behaviors, and edge cases.
  • Iterate in small steps: Commit changes frequently. It’s easier to roll back and diagnose issues when your history is granular.

Conclusion: Progress, not perfection

This live debugging session reminded me of two things:

  1. Agentic tools like Copilot and Playwright MCP can genuinely accelerate troubleshooting. Especially when you provide the right context.
  2. Describing requirements is hard. And that’s okay! Iteration, feedback, and even a few missteps are part of the process (both in terms of learning, and solving bugs).

If you’re navigating similar challenges, dive in, experiment, and remember: progress beats perfection.

  1. Update Copilot custom instructions files for your repo. If you’re using Copilot coding agent, make sure to configure the copilot-setup-steps.yml.
  2. Install and start the Playwright MCP server in VS Code to provide Copilot access to a browser for UI testing.
  3. Describe your bug or feature clearly in a new chat with Copilot agent mode.
  4. Let Copilot propose and apply fixes. But always review code changes and test results.
  5. Iterate on requirements based on what you see. Clarify as needed. Make sure you’re being clear in your requirements too.
  6. Commit frequently! Work in a branch and save your progress at each step.

Are you using the Playwright MCP server for UI testing? Or maybe you have another favorite MCP server? Let us know — we’d love to hear how you’re using Copilot agent mode and MCP servers as part of your development workflow!

In the meantime, mark your calendars for our next Rubber Duck Thursdays stream, subscribe to us on YouTube, and check out skills.github.com for interactive GitHub learning. See you next time!

Learn how to set Copilot coding agent up for success with custom instruction and Copilot setup steps >

Written by

Debugging UI with AI: GitHub Copilot agent mode meets MCP servers

Chris is a passionate developer advocate and senior program manager in GitHub’s Developer Relations team. He works with execs, engineering leads, and teams from the smallest of startups, established enterprises, open source communities and individual developers, helping them ❤️ GitHub and unlock their software engineering potential.

Source link