Custom Framework for iOS

This entry is for my future reference.

In iOS & MacOSX development, a framework is basically a folder contains headers, nib files, images, object files and other files. Apple provides a list of frameworks along with iOS SDK. Unfortunately, Apple does not provide (or intentionally hided) the ability for Xcode to create custom iOS framework. Without custom framework, it will be troublesome to provide pre-built libraries with UI to external parties. Fortunately with some tricks/hacks one can create a static link custom framework. Please refer to following references.

References:

• http://db-in.com/blog/2011/07/universal-framework-iphone-ios-2-0/
• http://jaym2503.blogspot.hk/2012/09/how-to-create-custom-ios-framework.htm

Releasing AVPlayerLayer and AVPlayer

While using AVPlayerLayer and AVPlayer together, I found that the retainCount of the AVPlayer object is not decremented immediately upon destroying the associated AVPlayerLayer. It seems to be scheduled some time later. If I release them in applicationDidEnterBackground: the free-up progress will just stopped (and the app entered into background) before the freeing is completed. This caused some problems in my project. After some searching, I found that beginBackgroundTaskWithExpirationHandler: allows running background task and so it gives time for the non-finished free-up task. I simply adding it to applicationDidEnterBackground: and the free-up is done completely.

- (void)applicationDidEnterBackground:(UIApplication *)application
{
    UIBackgroundTaskIdentifier identifier = UIBackgroundTaskInvalid;
    identifier = [[UIApplication sharedApplication] beginBackgroundTaskWithExpirationHandler:^{
        [[UIApplication sharedApplication] endBackgroundTask:identifier];
    }];
}

Update: After some searching and trying, I found that a better way is to listen to UIApplicationDidEnterBackgroundNotification event in my class, and I do not need to use the background task hack anymore then.

[[NSNotificationCenter defaultCenter] addObserver:self
                                         selector:@selector(playerDidEnterBackground:)
                                             name:UIApplicationDidEnterBackgroundNotification
                                           object:nil];

Using NSFileHandle.writeabilityHandler

I am using [NSFileHandle writeData:] to send data over network in one of my projects, and I found an interesting problem when I run the app with the testing device, which is using iOS 5.1. After 100 bytes of data are sent, the writeData method halts and throws an exception. Results from Google search told me that it is a problem since iOS 5.0, and maybe I should use NSFileHandle.writeabilityHandler, which is introduced since iOS 5.0 / MacOSX 10.7, instead. Unfortunately, I could not find a single example on web about how to use NSFileHandle.writeabilityHandler should be used. After a day of trial-and-error I finally figured out a workable solution, and I will put it here as a reference. However, I must say this solution is only workable, but not elegant, not even close. I believe there must be a better solution than mine out there.

Update:

• It seems that fileDescriptor can be used directly in send() or sendto()
• It is better to use send() which will not block this thread
• Updated the code to end the sending if there is sending error

// Some definitions
BOOL isAtOrAboveIOS5 = [yourfileHandle respondsToSelector:@selector(setWriteabilityHandler:)];
NSMutableData* pendingData = [[NSMutableData alloc] initWithCapacity:1024];

// Try to send data here
// iOS 5 / OSX 10.7 or above
if (isAtOrAboveIOS5 == YES) {
    [pendingData appendData:dataToSend];
    remoteFileHandle.writeabilityHandler = ^(NSFileHandle* thisFileHandle)
    {
        int amountSent = send([thisFileHandle fileDescriptor], [pendingData bytes], [pendingData length], MSG_DONTWAIT);
        if (amountSent < 0) {
            // errno is provided by system
            NSLog(@"Error while sending response: %d", errno);
            amountSent = [pendingData length];
        }
        [pendingData replaceBytesInRange:NSMakeRange(0, amountSent) withBytes:NULL length:0];

        // Finishing
        if ([pendingData length] == 0) {
            thisFileHandle.writeabilityHandler = nil;
        }
    };
} else { 
    [yourfileHandle writeData:dataToSend];
}

My guess of the actual problem when using writeData: after iOS 5.0 is that writeData: just tries to send as much data as possible over the socket in one shot. If there is much data than the socket can accept (this time), exception is thrown. With NSFileHandle.writeabilityHandler set, the code block is executed everytime its fileDescriptor (this time it is a socket) can accept more data. Since there is no method in NSFileHandle which tells you how many data is actually written, I used function sendto() here, but I believe there should be a better implementation exists.

Updating UIView inside function observeValueForKeyPath:ofObject:change:context:

This time, I try to observe change of a value using KVO and update UI after a specific change, in iOS. I put the code of UI updates inside observeValueForKeyPath function like the following:

- (void)observeValueForKeyPath:(NSString *)keyPath
                     ofObject:(id)object
                       change:(NSDictionary *)change
                      context:(void *)context
{
    if (object == targetObject && [keyPath isEqualToString:@"targetKeyPath"]) {
        if (targetObject.targetKeyPath == targetValue) {
            // self is a pointer of UIViewController
            [self.view setBackgroundColor:[UIColor blackColor]];
            NSLog(@"UI Update Here!!!");
        }
    }
}

It turns out the background color does not change as planned but the log is printed. I am sure the UIViewController is the top view controller so it is not view-controller-is-not-top-view-controller-problem as some Google results teach me. I was frustrated and tried my best to Google for a few hours without luck. Finally, an example in iOS Developer Library gave me a huge hint. In that example, its author put this line of code inside observeValueForKeyPath function, just before lines which will update a table view.

assert([NSThread isMainThread]);

I placed this line in my code and the assertion failed. I couldn't help but think: Is there any relationship between "Updating UI" and "Main Thread"? It turns out to be a huge YES. Check out this paragraph in UIView Class Reference.

Threading Considerations

Manipulations to your application’s user interface must occur on the main thread. Thus, you should always call the methods of the UIView class from code running in the main thread of your application. The only time this may not be strictly necessary is when creating the view object itself but all other manipulations should occur on the main thread.

At this point, the root cause of the problem becomes clear:

1. The observeValueForKeyPath function does not run in main thread. (Maybe it does run on the main thread sometimes, but we have no control anyway)
2. Any UI manipulations must occur on the main thread

Knowing these, solving the problem becomes easy:

- (void)observeValueForKeyPath:(NSString *)keyPath
                     ofObject:(id)object
                       change:(NSDictionary *)change
                      context:(void *)context
{
    if (object == targetObject && [keyPath isEqualToString:@"targetKeyPath"]) {
        if (targetObject.targetKeyPath == targetValue) {
            // self is a pointer of UIViewController
            [self.view performSelectorOnMainThread:@selector(setBackgroundColor:)
                                        withObject:[UIColor blackColor]
                                     waitUntilDone:NO];
            NSLog(@"UI Update Here!!!");
        }
    }
}

Finally it works like magic :)

Lock-free Queue (Single Producer Single Consumer)

I was searching for an efficient thread-safe queue implementation last week for my project at work, and I came up with the term "Lock-free Queue". It seems to be already a widely studied topic (Google suggested me a few papers) but I think they are too difficult to understand. After that I found this page which introduced and explained the author's lock-free queue implementation. Although it has certain limitations (only one producer and one consumer are allowed), its implementation is very simple and I think the author's explanation is simple and clear. What's more, it fits my case (even with those limitations).

For my own reference, I put part of my code (a little bit modified from what's described by the author) below. With CUSTOM_QUEUE_NODE_REUSE flag, the enqueue function will not do the lazy removal as the author suggested. Instead it directly reuses nodes which were previously dequeued. The drawback of this modification is the actual memory usage of the queue will always be the same as when it reached its longest queue size.

Update: Since C++ increment and decrement operations are not atomic and it causes bug in the code below, I removed the related code. If one needs to deal with queue size, he has to find atomic functions for increment and decrement.

Working with JSON using C# (.NET 3.5 or above)

Using .NET framework 3.5 or above, one can manipulate JSON object using a JavaScriptSerializer. To do so:

1. Adding .NET reference System.Web.Extensions
2. Use namespace System.Web.Script.Serialization
3. Create appropriate classes to describe the target JSON structure

The following example demonstrates how to read from and write to a JSON object.

Note: The JSON string used below is adopted from web.

P.S. In C#, two double quotes are used to describe one double quote in a multi-line literal (string), all escape characters (e.g. \") does not work in this case

OSX Lion Boot Options

This is a reminder for myself~

Once Mac OSX Lion starts booting up, you may press the following key combinations for analyzing and/or fixing your system:

• Command + V => Verbose mode, so you can check what makes your system wait or which error makes it stuck.
• Command + R => Recovery mode, you can access disk utilities to check and/or fix your disk, access the Internet for solution, and reinstall your OSX.
• Option => You can select which disk to boot
• Command + S => Single user mode, the system will boot into terminal mode and login as root. You can also try to check and fix your disk here, via commands (fsck, fsck_hfs etc.)
• C => Boot CD/DVD rom.

Programming with Windows Screensaver

While developing your Windows application/system, you may want to disable/enable/deactivate the screensaver. Here are some ways to do so: (ref: http://support.microsoft.com/kb/140723)

If the screensaver is activated (because halting for some time) and you want to kill it (activate the machine), you may use the following C code:

To temporally disable screensaver (maybe your application is working on something and you do not want the screen to turn off automatically), you may use:

However, your system may go wrong using the above code. Imagine if your application crashed after disabling the screensaver, then it will keep disabled until you re-enable it manually. In this case, you can just set the state of your current thread to tell the system that this thread requires attention.

Android Kill Self-process

I came across a line of source code written by some others today, and it gives me a behavior I have never thought about. The whole application (simplified version) consists of two activities. A user first launches up the main activity and clicks a button in this activity. This starts the second activity. The user then clicks a button in the second activity, and this line is called:


I thought the whole application will be killed and the system will go back to home screen but it does not. Instead the application goes back to main activity and calls its onCreate() again. I am surprised, and then I did a test. I added finish() in main activity after startActivity(). This time, killing the process just closed the whole application.

I tried to search for the reason of this behavior but I cannot find out the answer. Normally, processes are managed by Android system and one should not handle them himself. I believe this behavior is because some resource is not freed yet when the process is killed, and the system thinks the application should relaunch (or it thinks the application is still launching for the first time?).

Some Notes about GINA

GINA (Graphical Identification and Authentication) is a component which works with Winlogon to provide secure authentication and interactive logon services for Windows before Windows Vista. GINA is implemented as a replaceable dynamic link library (DLL), so that one can customize it by writing his own DLL. For GINA basics, please read them here. Below I would like to make some notes about problems I met while developing a customized GINA. I will expand the list when I find something new.

1. Do not call WlxSasNotify() to notify Winlogon for a SAS (Secure Attention Sequence, normally Ctrl+Alt+Del) in a thread different from the GINA thread. In my case, I post a message using PostMessage() SendMessage() to the SAS dialog box and call WlxSasNotify() there. Otherwise, the machine failed to shut down after recovering from hibernate.
2. Normally, Winlogon will capture a SAS appearance and call appropriate GINA functions. If you are replacing it with something else (so you are not triggering SAS manually), I have found that Winlogon may fail to call those GINA functions and results in halting between logon and logoff/locked states. Therefore you should notify Winlogon using WlxSasNotify(). However, you cannot just call WlxSasNotify() in anywhere, otherwise you may trigger some other abnormal behavior. After trial and error, I found that only calling WlxSasNotify() from the dialog box gives out normal results.

Update: Using PostMessage() will trigger another bug (cannot shut down after recovering from standby)

Static Class Member Variables in C++

Recently when I tried to write a class with static member functions and variables, I found a strange linking error. No matter how I tried, the linker did not seem to be able to link those static member variables properly. After some searching, it seems to be a common compiler "feature" of C++....

In order to define the variable, define it outside the class definition, but the declaration is still inside the class.

Observations of MIFARE Classic 4K Card

Recently I am developing an application for Windows environment with smartcard, and below are some notes I would like to make about MIFARE Classic 4K card:

• Block 0 is not writable. It contains card ID and information stored by manufacturer
• Blocks are divided into sectors, and each sector contains one trailer block
• Trailer block stored keys to authenticate the sector it belongs and attributes which decide access right to the sector
• Handling the trailer block not carefully may lead to all rights to that sector disabled, and I have successfully (or I should say accidentally...) locked the whole sector 0. However, as the GetID command is still functional, that command should be using some other way to read the ID from block 0 / some hidden block?
• As trailer block stores the authentication keys of its sector, updating them means changing the authentication keys. Location of keys: first 6 bytes -> Type A; last 6 bytes: Type B
• Reading/Writing a data block requires authentication beforehand
• Authenticating one block is equivalent to authenticate the whole sector
• Only one sector can be authenticated at any time. E.g. If you authenticate sector 2 after authenticating sector 1, you can now only access sector 2 but not sector 1. In order to access sector 1, you have to authenticate sector 1 again.
• The above does not only apply to one single application. i.e. If application A authenticated sector 1 and then application B authenticate sector 2, application A will not be able to access sector 1 without authenticating sector 1 again.

Using sysinfo in Android NDK

Sometimes you may need to write a program which needs to query some system information (e.g. system uptime). In this case, one may want to call the sysinfo() with struct sysinfo as defined in <sys/sysinfo.h>. Unfortunately, Android NDK does not provide the sysinfo() definition (definition is included since android-9, but I am still having no luck to link it properly). After some searching, I found this thread provides a solution: adding an assembly source (.S). Build it with your project and it works like a charm. The assembly source is attached below for your reference.

STLport in Android NDK r5b

Android NDK includes a special version of STLport since r5, and let developers to use it by specifying a flag in Android.mk files. However, if you are developing using Makefile instead (like I do), using STLport may be a little bit complicated.

First, since the build-standalone-toolchain script of NDK (also included since r5) can only include stdc++ library into the toolchain it builds, compiling with STLport headers may lead to a list of build errors. In order to prevent this, using prebuilt compilers in the NDK ("the hard way" as described in readme) may be a better choice. In this case, you will have to take care with sysroot.

Second, developing with this special STLport has some limitations as mentioned in readme: Exception and RTTI are not supported. Remember to build every cpp source files with -fno-exceptions and -fno-rtti.

Adding NTFS Write Functionality to OSX

Okay the story is like this: I have Windows, Ubuntu and MacOSX systems in my home and office and sometimes I backup or transfer files using a 8GB USB thumb drive (another way is to use Dropbox). This works perfectly fine until one day I met a 4.x GB file and OOPS! FAT32 filesystem does not allow a single file with that size! As a result, I tried to search for a mulit-platform supported filesystem which also supports large files. I tried NTFS. It works fine with Windows and Ubuntu but it can only be read from MacOSX, without write... When I was going to give up, a friend suggested this Lifehacker article to me (thanks again to my friend!). According to that article, I added NTFS write function to the MacOSX system and now my problem is solved~

The steps are simple:

1. Download and install MacFUSE from here
2. Download and install NTFS-3G for Mac OSX from here
3. Restart your Mac
4. Enjoy~

Using Thread in iOS

iOS SDK has its own NSThread (class reference here) class to handle thread operations, and using it is pretty simple:

In the above source, func_to_call is the function to be performed in the thread and parm_to_func (can be nil) is the parameter to be passed to func_to_call. If one would like to call functions in the main thread from child thread (probably you want to do this when the child thread is going to end and notifying the main thread is required), using the following:

In this line, func_in_main is the function to be called in main thread.

Note: The above method should be compatible with iOS 3.x.

Macbook Pro cannot boot up with continuous beep and black screen

Today my colleague told me that his 13" macbook pro (2009 ver.) cannot boot up with black screen. After taking a look, I found that it even gives out crazily annoying beep sound continuously (beep-beep-beep, halt, beep-beep-beep, halt, ......). I have not met that before and turned to Google for help. After some looking and trying (at some time I thought it is because a disc stuck in the drive), finally I found a few sources all point out that the 3-beep sound means "bad memory". The colleague then tried taking the two 1-GB RAM in and out to test and BINGO! The problem was caused by one single faulty RAM! So I will know what to do if any of my machine gives me that sound......

Update: One of my friends told me just now those beep sounds are called "beep code" and their meanings are dependent to different BIOS. A reference of Mac machines can be found here.
Update: Another reference in Wikipedia.

Notes of differences between x86 and x64 on programming

Notes: For reference only, actual situation should be affected by CPU type, kernel, compiler, etc.

Data Type	Ubuntu x86	Ubuntu x64	Win32	MacOS 10.6	SunOS 5.10	MIPS	iOS
char	1	1	1	1	1	1	1
short	2	2	2	2	2	2	2
int	4	4	4	4	4	4	4
long	4	8	4	8	4	4	4
long int	4	8	4	8	4	4	4
long long	8	8	8	8	8	8	8
float	4	4	4	4	4	4	4
double	8	8	8	8	8	8	8
long double	12	16	8	16	16	8	8
pointer	4	8	4	8	4	4	4

Mathematical Prove (3)

Bilinear Pairing over Composite Group

Given groups $\mathbb{G}$ and $\mathbb{G}_T$ of order $N=pq$ where $p,q$ are prime, and a bilinear pairing $e: \mathbb{G} \times \mathbb{G} \rightarrow \mathbb{G}_T$.

The subgroups of $\mathbb{G}$ with size $p,q$ are defined as $\mathbb{G}_p$ and $\mathbb{G}_q$. Let generator of $\mathbb{G}$ be $g$. The generators of $\mathbb{G}_p$ and $\mathbb{G}_q$ be $g_p$ and $g_q$ can be computed as $g_p = g^q \in \mathbb{G}_p$ and $g_q = g^p \in \mathbb{G}_q$

In addition to basic bilinear pairing properties, bilinear pairing over composite group holds some additional properties:

• $e(g_p,g_q) = e(g^q,g^p) = e(g,g)^{pq} = e(g,g)^N = 1$
• $e({g_p}^a{g_q}^b,{g_p}^c) = e(g^{qa+pb}, g^{qc}) = e(g,g)^{q^2ac+Nbc} = e({g_p}^a,{g_p}^c)$

Attribute-based Broadcast Encryption

Attribute-based Broadcast Encryption (ABBE) makes use of Attribute-based Encryption scheme to implement broadcast encryption. In attribute-based encryption, a ciphertext is bundled with a data access policy (policy for short) which consists of a list of attributes connected by logic gates (typically AND and OR). Each user in the system possesses a list of attributes. To determine if a user is a legitimate decryptor of the ciphertext, the system compares his attributes with the policy. If the policy is satisfied, the user is able to decrypt the ciphertext.

Compared with conventional broadcast encryption, attribute-based encryption is like separating users into different groups and the access control (using policy) is enforced by manipulating access controls in those groups. If the number of attributes is much less than the total number of users in the system, ABBE has a big advantage on efficiency. However, it cannot enforce ad-hoc access control on every individual user like conventional broadcast encryption.

Hence, some proposed using each bit of user ID as attribute, which makes the total number of attributes becomes log N (assuming N users). This number is much smaller than the total number of users and it sounds promising. However, during my study I found two major problems in this scheme when the number of legitimate users grows large.

First, in order to get a small ciphertext size, the scheme uses Boolean algebra simplification algorithm. The idea is to first list all user IDs of legitimate users in a truth table, then applies the simplification algorithm to get a shorter Boolean algebra expression without altering its result, and this problem is NP-Hard. The two most famous algorithms for this task are Quine-McCluskey algorithm and Espresso heuristic logic minimizer. The complexity of them is yet to be found, but according to my preliminary testing, the running time (for just Boolean algebra simplification) can be much longer than running encryption of Boneh's broadcast encryption. A better simplification algorithm or a better implementation of it is required.

Second, the number of sum-of-product terms output from the simplification is not a constant. In other words, the ciphertext of this scheme will have a variable size. And this leads to a question: How large can the size be? I found it difficult searching for or understanding literature which discusses about this problem. Therefore, I try to run my own test again. I tried to compute both average case and worst case on small N (since I have to generate all combinations, it is already very large even for small N, e.g. 32). From my test, the growth of average size seems to be O(log N). For worst case, however, the size seems to be N/2. In my opinion, the problem is how often the size stays at O(log N). In real application, I think a good average bound is not enough, a good worst case bound is needed. Imagine when N = 10M, N/2 will be 5M and it results in a enormous ciphertext size.