Calling fopen_s/_wfopen_s with ccs=UNICODE flag

Just recently I had a discussion with my co-workers about my program that has to interact with Web servers via HTTP post. In my program I post a file and I told my co-worker that the file is in the format of UNICODE.

However, when he tries to decode it, he found that the file is actually not in the format of UNICODE especially not in UTF-16 but in the format of ANSI. So I said to him that it is not possible because I create a file by explicitly specifying that the file should be in the format of UNICODE. But close examination of my file showed that the file is indeed in ANSI format. So I quickly looked at my code where I create a file.

Here is the code.

_wfopen_s(&fp, filename, L"w+, ccs=UNICODE");

Then, I looked up MSDN and found the following.

Allowed values of the encoding include UNICODE, UTF-8, and UTF-16LE. If the file is already in existence and is opened for reading or appending, the Byte Order Mark (BOM) is used to determine the correct encoding. It is not necessary to specify the encoding with a flag. In fact, the flag will be ignored if it conflicts with the type of the file as indicated by the BOM. The flag is only used when no BOM is present or if the file is a new file. The following table summarizes the modes used in for various flags given to fopen and Byte Order Marks used in the file.

Encodings Used Based on Flag and BOM

Flag	No BOM (or new file)	BOM: UTF-8	BOM: UTF-16
UNICODE	ANSI	UTF-8	UTF-16LE
UTF-8	UTF-8	UTF-8	UTF-16LE
UTF-16LE	UTF-16LE	UTF-8	UTF-16LE

As you can see above, if it is a new file and I specify UNICODE, it is actually creating a file in the format of ANSI. Obviously I missed that and I somehow thought that the file would be created in the format I specified.

So the lesson I learned from this incidence is that I should be more careful reading MSDN page. Lastly, here is MSDN link to fopen_s/wfopen_s: http://msdn.microsoft.com/en-us/library/z5hh6ee9(v=vs.80).aspx