WPP trace sample - USEPREFIX and USESUFFIX

Just recently I had to add some WPP traces and found that macro preprocessing makes it somewhat difficult to use it. I know that it has certain advantages but I did not find much information nor good samples for my own needs so I would like to share some of my findings.

Essentially, what I wanted to do is that I want to call some function to extract certain information from the arguments that I passed to WPP macro. The issue is that WPP preprocesses its own macros and produces another set of code before the build kicks in so and the other thing is that existing code already used debugoutput API so I wanted to minimize the code churn as much as possible. While working on this, I learned that I can use PREFIX and SUFFIX but I did not find any good samples.

Following code basically helps me to call foo() function with EXP parameter and SUFFIX will output whatever the information that I obtained from foo.
There is one thing to keep in mind.
When creating PRE and POST macros, you need to put parameter names as part of the names in the middle. Essentially, you can put a lot of code in PRE macro but let us remember that this is macro so your code size will grow pretty quickly and if you happen to use this in kernel driver, you will also need to be careful of APIs that you use in PRE to make sure that you follow IRQ level.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

//MACRO: MYTRACE // //begin_wpp config //USEPREFIX (MYTRACE, "%!STDPREFIX!"); //FUNC MYTRACE(LEVEL, EXP, ...); //USESUFFIX (MYTRACE, "Result=%!HRESULT! %s",EXP, pszId); //end_wpp #define MYTRACE(level, exp, ...) \ do { \ TraceEvents(level, VA_ARGS(__VA_ARGS__)); \ } while (0) #define WPP_LEVEL_EXP_PRE(level, exp) \ { \ PSTR pszId = foo(exp); \ #define WPP_LEVEL_EXP_POST(level, exp) ;} #define WPP_LEVEL_EXP_ENABLED(LEVEL, HR) WPP_FLAG_ENABLED(LEVEL) #define WPP_LEVEL_EXP_LOGGER(LEVEL, HR) WPP_FLAG_LOGGER(LEVEL)

Once you have the above defined, you can then start to call your WPP macro as follows.

1	MYTRACE(TRACE_LEVEL_ERROR, hr, "hello, world");

This way, we can be more flexible using WPP traces. Hope this helps someone.

Studying for PowerShell and Regex

At work, PowerShell with Regex is used for log search and hence, I am putting hours to learn these two together. I look at this as a good opportunity to learn these two as this is something that requires time to practice and having opportunities to use these two are great ways to learn them.

I tried to learn PowerShell a few times but never really reached a level where I felt comfortable but at this time, I am reading through Windows PowerShell Cookbook and finding that PowerShell has gone through many changes and many new commands.
For instance, I just ran into Invoke-WebRequest whose aliases are 'iwr', 'curl', or 'wget' I remember that we had to create new object using System.Net.WebRequest but with this new command, I don't see the needs. Good addition! In fact, there are many new commands and their alias makes it easy to transition to PowerShell world and as for me this script from the book really helped me to learn alias as I learn PowerShell.
But the most surprising addition to the latest PowerShell is readline library. With readline library, you can turn cmd window into something completely different. One of PowerShell author at Microsoft has implemented really nice module based on this PsReadline library and you should get it from here if you have done so. This will make PowerShell learning quite pleasant.

For regular expression we all know that it is more powerful and yet complicated than wildcard matching. I've looked through a book and some MSDN site but I have found Mastering Regular Expression much more complete than any other sources. So I picked up this book and practice it via PowerShell interface. In PowerShell we can run Regex using "-match" operator in PowerShell. Here is an example.

 PS C:\Temp> "12" -match "\b([1-9]|1[012])\b"  
 True  
 PS C:\Temp> "15" -match "\b([1-9]|1[012])\b"  
 False  

At first, I did not use "\b" which is word boundary. Without this it was producing incorrect result. I won't explain all the details of Regex here as I don't know enough but again point you to the Mastering Regular Expression. ;-)

If anyone is interested in learning these two, try these two books. ;-)

Different ways of using windbg: Learn the code flow and verify the change.

Recently I switched to a different group and hence, I've been trying to ramp up quickly and started to work on bugs to learn the product.
While working on bugs, I learned that debugger is really helpful to understand the code flow and verify the change. Typically, people might use debugger to debug the issue and that's correct. However, there is another advantage of using a debugger. That is to learn how the code flows and verify the change you are making.
What do I mean by that?
First, let me illustrate with an example to show how debugger can be helpful to learn the code.
Say, you understand what the certain function does and noticed that this function is called several places but not sure exactly where the call is coming from and where it goes.
Then, set a breakpoint on that function and try to reproduce the issue. Sure enough, this will hit the function but at the same time, you might be surprised to find how this function is called from different places for other purposes.
As an example, we are learning NtCreateFile function and wanted to see how this is used. Then, set a breakpoint at NtCreateFile and see the callstack as follows:

 0:000> kcn8  
  # Call Site  
 00 ntdll!NtCreateFile  
 01 KERNELBASE!CreateFileInternal  
 02 KERNELBASE!CreateFileW  
 03 KERNEL32!GetDefaultSortFileMapping  
 04 KERNEL32!SetupDefaultSortTables  
 05 KERNEL32!InternalSortGetHandle  
 06 KERNEL32!SortGetHandle  
 07 KERNELBASE!GetSortVersionHandle  

As a trick, you can also give certain commands to breakpoints. For example, you can say, each time we hit NtCreateFile, display callstack and continue.

 bp ntdll!ntcreatefile "kcn; gc"  

Or if we are only interested in NtCreateFile() coming from certain function like CreateFileInternal as in the above example, we can do the following trick.

 0:000> bl  
  0 e 00007ff8`9b641bc0   0001 (0001) 0:**** ntdll!NtCreateFile "bd 0; x; gc"  
  1 e 00007ff8`98af72d0   0001 (0001) 0:**** KERNELBASE!CreateFileInternal "be 0;gc"  

Basically, what we are doing here is that we enable breakpoint #0 when we hit breakpoint #1 and as soon as we hit breakpoint #0, we display local variables and disable itself. This way we can suppress any other noise and quickly understand the code flow for your purpose.

Now that you understand the code and came up with the fix but sometimes it might be pretty difficult to reproduce the issue. Sometimes we can use debugger to change the code flow.
For instance, you have a function that returns boolean value and in order to hit the newly added code path, you need true from the function but you can't really reproduce the issue locally. In that case, you can again take advantage of breakpoint.
Here is how. Say you have a function and try to disassemble the function and see the address of 'ret' instruction. Once you find it, you know that the return value is passed via eax register so you can do the following.
Here is the disassembled code:

 notepad!IsTextUTF8+0x3c:  
 00007ff6`59596e0e 488b5c2408   mov   rbx,qword ptr [rsp+8]  
 00007ff6`59596e13 c3       ret  

Here is breakpoint:

 0:000> bl  
  2 e 00007ff6`59596e13   0001 (0001) 0:**** notepad!IsTextUTF8+0x41 "r eax=1;gc"  

Of course, this may not work all the time but sometimes this could be very handy to verify your change quickly.

In summary, a debugger can be very helpful in learning the code and verifying the fix. Spend some time learning how to use the debugger. It can save much of your time.

PowerShell commands to configure Generation 2 VM kernel debug

Generation 2 VM has UEFI rather than BIOS. This means that secure boot is enabled by default but kernel debugging and secure boot are mutually exclusive.
Therefore, we will need to disable that first. Generation 2 VM still supports serial ports. We can set up serial ports for debugging again through PowerShell.

Here are two PowerShell commands to accomplish this:

 PS C:\> Set-VMFirmware -VMName testvm -EnableSecureBoot Off  
 PS C:\> Set-VMComPort testvm 1 \\.\pipe\vmdebug  

Switching to a different group - How I prepared for it.

I have been working in my current team for about three years so far and wanted to change the team to do different type of work. While I enjoyed debugging some hard problems and coming up with the fix, I have been wanting to consistently work on certain code base to learn the technology and deepen domain knowledge. Hence, I reached out those who I used to work with to gather some input/advice before I start to look around and today's blog is about what I have learned/gathered during this process.
First, do I really want a change?
One person whom I spoke to said it very clearly. He said that he could have made more impact on the product if he had stayed in my current team but he knew that he's not learning new technologies as much as he wanted in the current team. As such, he made a move and that has challenged him a lot for the past one year and felt that he grew so much. His new role/work is involved in big data/scale out work which helps him to be exposed to the latest hot topics and he's enjoying what he's doing. He mentioned that the process has been somewhat painful but that was intentional and he's glad that he made the move. As I was listening to him, I felt that I am in the similar situation that he was in a year ago and I want to have the similar changes in my own career. After that, I reached out to few more guys that I used to work with to gather some more information on how they found their new jobs. This helped me to what to look for and what's ahead when looking for jobs. With this, I started to look around opportunities inside the company mostly and started to send out my application with my resume based on HR job lists.

Second, what should I prepare for the interview?
You might say, "Interview?" Yes, we may still need to go through the interview process even if we change jobs inside the company. I learned that if I change jobs within the same group, I don't have to go through the interview but if I want to switch to different group, the interview process is required. As you guessed, preparing for the interview is not an easy task in computer science field. You will have to grab those programming algorithm books that I used in school and try to brush up your problem solving skills. Sometimes I wonder if this is really necessary in that we may never get to use some of these algorithms/data structures. Yet, that's the way it is so you will need to adapt to the system for the time being. :) Thanks to my friends, I came to know two very useful resources to prepare the interview and they are leetcode.com and EPI(Element of Programming Interview).

These two helped me immensely to be ready for technical interviews so let me describe these two a bit more in detail. leetcode.com is very helpful in that it has online judge site where you can solve the problems and submit your code to run the solution against test cases to see if your solution is correct and efficient. On top of that, you can also peek into other people's solution in discuss forum to see if there is any better solution. From this site I was able to check if my solution is correct and also learned how others solved the problem to enhance my solutions. It has about 150 problems to solve and you will surely learn much and confirm your answer via this web site to get ready for the interview.
Now, let me talk about EPI. In my opinion, EPI is one of the best books out there to prepare for the technical interview. It has a lot of questions and these questions are not easy but through these questions I felt that I am getting ready for the interview. It has questions that I never thought about so that helps you to think about such problems so that your brain gets exercised. In fact, some of questions that I studied from this book came up during the interview so that was very helpful.
I know that there are many more problems out there and I have heard from friends that some other companies tend to ask more difficult questions than what I have found from the above two resources. However, I think that these two have to be the base to start with so I highly recommend them!

Additionally, I would like to emphasize the amount of time you want to spend to prepare. Personally, it is very helpful if you can concentrate in studying the above two within short period of time so that you do not lose the context. It might be difficult to find time to study but I feel that you should treat this preparation as part of your job and take it seriously. Probably, it is a good idea to study for about three to four hours each day and seven to eight hours during weekends so that you can make significant process.


Third, which company/team to choose?
I did not have a lot of choices as I had four different interviews total. While going through the interview and getting to know each team/company, it became clear to me which team/company I would like to join. I believe that the interview process is not just one sided. It is not just the interviewee trying to impress upon the interviewer and pass the interview. It is also the interview's responsibility to sell their team/company so that the interviewee would like to come and work with them. I interacted with four different team/company, I felt comfortable with one particular team and decided to join that team. To be clear, I did not get offers from all these four places I had an interview with.

Lastly and perhaps most importantly, I would like to mention the importance of networking.
Making a career change is not an easy task but I have received a lot of advice and tips from friends/ex teammates that I worked with. I feel that keeping these relationships well is very important aspect of our career and as such, my advice to everyone is to reach out to friends/ex teammates for advice and you will be surprised how much you can learn from your peers to succeed in making career change. So if you are thinking of making changes in your career, send an email to your friend for lunch and ask them for their input.

All the best!

Visual Studio makes it real easy to write drivers

I have heard sometime ago that visual studio now fully supports driver development. Years ago, I found some web resource to modify visual studio settings to make this happen but now that it officially supports driver development, I decided to give it a try.

In order to do this, you will need to install WDK from this msdn web site. Once you install WDK, start up Visual Studio and you will find Driver project when you start new project.
Not just that! In Visual Studio, you can even deploy your own driver to the remote computer!

MSDN also provides some hello world driver to help you to jump start.
Before deploying make sure to enable testsigning on remote computer and restart the remote computer.

However. if you want to do it manually, you will have to first install test certificate and after that right click inf file and click 'install'. That will install and start up your driver.

In case you have trouble installing driver, please check setupapi.dev.log file under C:\Windows\inf folder. If you have to debug your driver, then windbg is your friend.

Enabling PowerShell remoting

I am experimenting PowerShell remoting lately and wanted to share simple tip to enable remoting.
To enable remoting, you will want to run 'Enable-PsRemoting' cmdlet and once you run it, it asks you several questions to which you will probably want to answer 'yes'. If you want to skip this, you can run 'Enable-PsRemoting -force' and that skips question steps.

Once you have the machine ready, by default it only allows admin to execute the script so you might want to add yourself so you can execute commands from remote computer.
To add yourself, run 'Set-PSSessionConfiguration Microsoft.PowerShell -ShowSecurityDescriptorUI' cmdlet and UI will pop up and you can use that to add yourself.

Now that you have enabled remoting, let's give it a try:

PS C:\Users\ilhoye> invoke-command -computer localhost -scriptblock {ps | select -first 5 | ft}

Handles  NPM(K)    PM(K)      WS(K) VM(M)   CPU(s)     Id ProcessName
-------  ------    -----      ----- -----   ------     -- -----------
    124      11     2368       6068    46            2192 Acmengine
    890      42    16560      44128   150            3416 CcmExec
     46       7     1820       5140    57     0.19    532 conhost
     46       7     1844       5184    57            1664 conhost
     46       7     1824       5168    57     0.23   2844 conhost

Finally, if you still have a trouble setting up remoting, a good place to start debugging is to run 'get-help about_Remote_Troubleshooting'. This has detailed information on most issues you might encounter.