Search This Blog

Saturday, June 11, 2011

Priority Queue

Lately I have been studying Jon Bentley's book "Programming Pearl" and in the book it talks about priority queue with heap data structure. So I grabbed another book "Algorithms in C by Robert Sedgewick" and read more about it as it sounds pretty useful for many different applications.

According to these two books, a priority queue is a data structure to insert data and retrieve the item with either largest key or smallest key depending on your heap setup. We can implement priority queue using many different data structures but heap seems to be very good candidate. I borrowed two fundamental heap functions: siftdown() and siftup(). siftdown() sifts the first element down the data structure until it has either no child or it is larger than its children node and siftup() does the opposite.
So decided to implement it based on these two books and even though I referred the books, it was still somewhat tricky to get it right.

Tricky part was that I have to take care of a node with one child during siftdown(). At first I tried compare the node with both child in my while() loop but that did not work well. So referred the Algorithms in C for help and learned this edge case.

Followings are the code based on the two books I mentioned above. It needs more work if I need to use it for real but this should at least help me to get going.

typedef int node;
static node *pq;
static int N;

#define swap(a, b) \
do { node tmp = *a; *a = *b; *b = tmp; } while (0)

class prioQ
{
private:
node *pq;
int count;
int max_num;

public:
prioQ(int imax)
{
max_num = imax;
pq = new node[max_num+1];
count = 0;
}
~prioQ()
{
delete pq;
}
int empty()
{
return (count == 0);
}
int insert(node v)
{
if (count >= max_num) {
std::cout << "Queue is full." << std::endl;
return -1;
}
pq[++count] = v;
siftup(pq, count);
return 0;
}
node delmax()
{
if (empty()) {
std::cout << "No more item in the queue." << std::endl;
return -1;
}
swap(&pq[1], &pq[count]);
siftdown(pq, 1, count-1);
return pq[count--];
}

// helper functions from max heap
void siftup(node a[], int k)
{
while ((k > 1) && (a[k] > a[k/2])) {
swap(&a[k/2], &a[k]);
k = k / 2;
}
}
void siftdown(node a[], int k, int end)
{
int i;
while (k*2 <= end) {
i = k * 2;
// k*2 < N is to test one child node case
if ((k*2 < end) && (a[i] < a[i+1])) i++;
if (a[k] > a[i]) break;
swap(&a[k], &a[i]);
k = i;
}
}
};

By the way I used http://www.simplebits.com/cgi-bin/simplecode.pl?mode=process to copy my C++ code. It works great!

Sunday, June 5, 2011

sleep() does not put the process to sleep

Just recently I had to deal with the code that calls sleep() and suddenly I wondered if sleep will put the entire process to sleep. So I quickly checked man page and found the following.

SYNOPSIS
#include

   unsigned int sleep(unsigned int seconds);

DESCRIPTION
sleep() makes the current process sleep until seconds seconds
have elapsed or a signal arrives which is not ignored.


At first, I believed this man page but it just happened that I read Jefferey Rithcer's Windows Via C/C++ a few days ago and I remembered that on Windows there is no SuspendProcess() even though there is SuspendThread(). Jefferey explained that it is not quite possible to have SuspendProcess() in that Process can have many threads and these threads can come and go. Hence, it is pretty difficult to get all the threads to be suspended on Windows as thread is a scheduling unit.

Though I am looking at Linux for this particular case, it seems to me that I should check this out with other developers so I posted the question on stackoverflow(http://stackoverflow.com/questions/6192645/does-calling-sleep-from-pthread-put-thread-to-sleep-or-process).

Sure enough, I got an answer that my man page is outdated and I need to report to maintainer of man page to update the man page of my distribution. So the correct description of sleep is as follows. (taken from http://www.kernel.org/doc/man-pages/online/pages/man3/sleep.3.html)

SYNOPSIS
#include

   unsigned int sleep(unsigned int seconds);

DESCRIPTION
sleep() makes the calling thread sleep until seconds seconds have elapsed or a signal arrives which is not ignored.


So in conclusion, sleep() puts the calling thread to sleep not the process. Interesting!

Saturday, May 21, 2011

C# with ntdll call? Not so easy

Lately I started to study C# and this was due to the fact that I had to use it for my work. I have been using C most of the time so it is somewhat difficult for me to switch the language but I think that this is a great chance to learn the new language as it helps me to think differently when approaching the problems.

Anyhow, today I decided to try my C# to do things that are somewhat low-level and I know that C# is not designed for low-level tasks but wanted to try how it is like with C#. So I decided to get to Process Environment Block(PEB) and find the command-line arguments of a given process. In this particular example, I just used my own process but it is relatively easy to change it to point to other process by providing other process handle.

With that, let me show my code.

using System;
using System.Collections.Generic;
using System.Text;
using System.Reflection;
using System.Diagnostics;
using System.Runtime.InteropServices;

namespace PEB
{
class Program
{
enum PROCESSINFOCLASS : int
{
ProcessBasicInformation = 0,
ProcessQuotaLimits,
ProcessIoCounters,
ProcessVmCounters // omitted the rest
};

[StructLayout(LayoutKind.Sequential)]
public struct PROCESS_BASIC_INFORMATION
{
public int ExitStatus;
public int PebBaseAddress;
public int AffinityMask;
public int BasePriority;
public int UniqueProcessId;
public int InheritedFromUniqueProcessId;
public uint Size
{
get { return (6 * sizeof(int)); }
}
};
// following two definitions are from Windows via C C++
// check 'allow unsafe code' in properties page
[StructLayout(LayoutKind.Sequential)]
unsafe struct __PEB
{
public fixed uint Filler[4];
public uint InfoBlockAddress;
};
[StructLayout(LayoutKind.Sequential)]
unsafe struct __INFOBLOCK
{
public fixed uint Filler[17];
// Windows keeps this as Unicdoe(wide character)
// Without the following option, it only gets to the first character
[MarshalAs(UnmanagedType.LPWStr)]
public string cmdAddress;
}

// Finding NtQueryInformationProcess API from ntdll is relatively easy
[DllImport("ntdll.dll", SetLastError = true)]
static extern int NtQueryInformationProcess(
IntPtr processHandle,
PROCESSINFOCLASS processInformationClass,
ref PROCESS_BASIC_INFORMATION processInformation,
uint processInformationLength,
ref int returnLength);

static void Main(string[] args)
{
int status;
int returnLength = 0;
Process currentProcess = Process.GetCurrentProcess();
PROCESS_BASIC_INFORMATION pbi = new PROCESS_BASIC_INFORMATION();

status = NtQueryInformationProcess(
currentProcess.Handle,
PROCESSINFOCLASS.ProcessBasicInformation,
ref pbi,
pbi.Size,
ref returnLength);

if (status == 0)
{
Console.WriteLine("PEB address: {0}", pbi.PebBaseAddress);
// It is little tricky to do casting in C# - two steps
// First, it is necessary to get the pointer
// Second, cast that to what we need using Marshal.PtrToStructure
IntPtr pebAddress = (IntPtr)pbi.PebBaseAddress;

__PEB peb = (__PEB)Marshal.PtrToStructure(pebAddress, typeof(__PEB));
IntPtr infoblkAddr= (IntPtr)peb.InfoBlockAddress;
__INFOBLOCK infoBlock =
(__INFOBLOCK)Marshal.PtrToStructure(infoblkAddr, typeof(__INFOBLOCK));
Console.WriteLine("command line: " + infoBlock.cmdAddress);

}
}
}
}

I put comments on my code to explain what I am trying to do but basically at the end of the day, the lesson I learned from my exercise is that I should choose the right language for the job. I felt that C# is probably not the best language to work on low-level ntdll stuff. I think that for this kind of job C/C++ is better language. Perhaps it is just that I am still in the process so I am just not good.
Anyway, I am enjoying learning C# so hopefully next time, I will write something more meaningful and related to C#.

Friday, April 2, 2010

message queue open failed with ENOMEM

Yesterday I was working on message queue part and found that every now and then when I started a daemon process, it fails to create a message queue with ENOMEM error number.

What was interesting to me is that if I restart a daemon again, this does not happen. So I looked into this issue and found that when the daemon is down for a while and other clients keep sending messages, those messages get accumulated in the kernel after for a while. Then when we try to start a daemon process, my code simply unlinks message queue and tries to create(mq_open) a new one with the same name. When we do this, I think that all the messages that were in the queue are not removed quickly and thus, we fail to create a message queue due to memory limit. (We can find the memory limit by calling getrlimit with RLIMIT_MSGQUEUE)

This can happen no matter how small the message queue is and here is a way to view how our message queue is doing.

$ sudo mkdir /dev/mqueue
$ sudo mount -t mqueue none /dev/mqueue
$ sudo cat /dev/mqueue/afcollectorapi
QSIZE:1232 NOTIFY:0 SIGNO:0 NOTIFY_PID:0

Thus, what we should do is that we do not unlink message queue but simply open the existing one and consume all the messages there.

Friday, November 2, 2007

library level system call hooking

We can use LD_PRELOAD to load our own library before any other libraries are loaded.


// myopen.c
#include
#include
#include
#include
#include

int open(const char *pathname, int flags)
{
void *handle;
char *error;
int ret;
static int (*orig_open)(const char *pathname, int flag) = NULL;

if (!orig_open) {
handle = dlopen("/lib/libc.so.6", RTLD_LAZY);
if (!handle) {
fputs(dlerror(), stderr);
exit(1);
}
orig_open = dlsym(handle, "open");
if ((error = dlerror()) != NULL) {
fprintf(stderr, "%s\n", error);
exit(1);
}
}
printf("calling open for %s\n", pathname);
ret = (*orig_open)(pathname, flags);

return ret;
}


// test.c
#include
#include
#include
#include

int main()
{
int fd;

fd = open("test.txt", O_CREAT);
write(fd, "cool", 4);
close(fd);
return 0;
}

# gcc -Wall -fpic -shared -ldl -o myopen.so myopen.c
# gcc -Wall -o test test.c
# LD_PRELOAD=/tmp/myopen.so ./test

Saturday, October 6, 2007

My sister's visit


My sister came to visit me last week and here is the picture I took with my baby and her baby.

Many people thought that they are twins and indeed they look alike. ;-)


Friday, October 5, 2007

prayer meeting

I write a short article for a prayer meeting each week and here is the one from this week.

Scripture reading
For I am convinced that neither death nor life, neither angels nor demons, neither the present nor the future, nor any powers, neither height nor depth, nor anything else in all creation, will be able to separate us from the love of God that is in Christ Jesus our Lord. (Romans 8:38-39)

For the past two weeks my sister’s family was here in America with me and we had a good time. Yet, the departure date finally came and we were sad that we had to say Good-bye. This reminded me of the well-known explanation of love. In Greek, there are various kinds of love depending on the type of relationship. First, there is a parental/familial love. This is love parents shows toward their children. Second, there is a platonic love between friends. Third, there is a romantic love between husband and wife. Although these loves are great in many senses, we also see the shortcomings. For instance, parental love has its own limitation in that the love is exclusive and temporary. When I was young, I thought that I would live with my parents and sisters forever. Yet, my sister’s visit shows that there is a physical distance gap that limits parental/familial love in one way or another. In addition, someday my parents will depart from this earth to be with the Lord. In other words, even parental love can be limited by distance or time. Therefore, this past two weeks was a good lesson for me because I could realize that human love cannot fill the void in our heart. So where can find we love that is eternal and permanent? Today’s verses show us that we can find that love in Christ Jesus. God first demonstrated His love by sending His one and only son when we still hated Him. No matter what we do to Him, God still loves us and He will not change his mind. Thus, today’s verses describe that His love is so strong that nothing can separate us from His love. So what is the love of God like? His love is limitless. His love is unconditional. His love is abounding. His love is eternal. His love is sacrificial. In conclusion, when I first came to see Manhattan, I was amazed at the number of dogs on the street. Whether it is true or not, someone told me that people are lonely in the city so they keep dogs to fill the void in their heart. I believe that people do this because God is love and we are made in his image so it is impossible to live without love. However, we cannot find the love that can fill their heart other than the love of God. So this morning I have good news for you. You have found the love big enough to fill your heart. Please be encouraged and share this good news. God bless you!