Raw access to SMS/MMS database on Android phones

February 14, 2012, 6:46 am

≫ Next: Mac OS X on VmWare - Upgrading from 10.6.4 to 10.6.8

My HTC HD2 running on Android from SD card has stopped working recently - the phone simply refused to power up while the battery was still fine. There was nothing I could do except to migrate to a new phone and re-sync everything from my Google account. Restoring the SMS, however, was not an easy task as my latest backup using Sms Backup and Restore is more than 3 months old.

Fortunately my Android was running from SD card, which was still working fine, so I could still access the entire Android folder on the card. Using various Windows tools (see my previous article) such as Ext2Explore, I was able to read the contents of data.img and found the location of the database storing all text messages, among other things, at /data/data/com.android.providers.telephony/databases/mmssms.db. The database is apparently in SQLite format.

Nextcomesthe challenge of reading the database. For this I use SQLite Database Browser, and all the tables are listed nicely:

Opening the SMS table and I can see all messages:

(phone numbers and message bodies have been removed by me when taking the screenshot)

The address column stores the sender/recipient of the message. Column body storesthe message body (for MMS/SMS) while subject is for MMS only. The type column specifies the message type (sent, received) and can be 1 or 2. Interestingly, the service centre that sent the message, or SMSC, is also stored but not visible from the stock Messaging application. The time of the message is stored as a Unix timestamp format. To convert it to a readable date in Microsoft Excel, use the following formula, adapted from this blog, and format the cell as Date:

=(((TIMESTAMP/1000/60)/60)/24)+DATE(1970,1,1)

If you want to migrate the SMS to a new Android device, you may be able to do it by simply copying the mmssms.db database to the other device. If not, you can export the messages to a CSV file using File->Export Table as CSV File and import them to Android, Blackberry (see my CSV2IPD tool) or other platforms.

↧

Mac OS X on VmWare - Upgrading from 10.6.4 to 10.6.8

February 17, 2012, 7:32 am

≫ Next: "Scanner Error" on Xerox WorkCentre 3119

≪ Previous: Raw access to SMS/MMS database on Android phones

In my previous articles (this and this) I have shown you how to setup a working Mac OS X 10.6.4 system running on your PC using VmWare Workstation. For some time, the setup has been working perfectly fine for me, mostly for iOS development, since I don't own a Mac.

However, recently I needed to upgrade my xCode to version 4.2 in order to support the latest iOS SDK, and to my surprise, Xcode 4.2 requires at least 10.6.7 for installation.

Well, if you don't want to read the rest of this post, there is an easier way. It should be noted that most of xCode and iOS SDK will work just fine without the extra features in 10.6.7 - the minimum version requirement is just Apple's lazy way to reduce development and support efforts by forcing users to upgrade to their latest release. Following this trick, I opened my SystemVersion.plist located at System/Library/CoreServices, and changed the version number to 10.6.7 or higher:

After a reboot, the new version was shown in About This Mac and xCode installed and worked just fine! Notice that this would fail (xCode installs but behaves erratically) if your current system is too old, e.g. 10.5.x. You'll need at least 10.6.4 for the trick to work.

However, with some free time at hand, I still decided to go ahead and properly update my system to 10.6.8 using the built-in Software Update. The updated system rebooted just fine, showing 10.6.8 correctly in About This Mac. For complete functionality, I installed the following 2 patches and rebooted the system again:

1. IOUSBFamily-378.pkg: allow the virtual OS to recognize USB devices. You would first need to disconnect it from the host and connect to the Mac via VMWare's Removable Devices menu.
2. EnsoniqAudioPCI_v1.0.3_Common_Installer.pkg: allow audio to work.

In my case, I also needed to disable the screensaver and various power management settings. For some reasons, it causes random kernel panic.

Next, I realized that there was not enough disk space to install the latest iOS SDK. Expanding the hard disk is not an easy task, as common methods fail with a "MediaKit reports partition map too small" error message. In my previous tutorial, I have shown a workaround which took a long time and did not always work. The proper method is to manually modify the partition table to reflect the new hard disk size. Run gpt tool from terminal as described in this post (see also this), destroy the existing partition table and recreate a new one with the correct partition size.

Take note that for gpt to work, you need to have the disk unmounted which is not easy as it's a system disk. You would need to boot from the install DVD and use Terminal from there. If a boot disk is not available, make a copy of the working virtual machine, insert the original hard disk as another drive, boot from the copied virtual machine and use it to work with the original hard disk. If you choose the second method, take note that the copied hard disk and the original hard disk will have the same UUID, which will cause problems with Disk Utility and VmWare will also warn you upon startup. The correct way to fix this is to use a hex editor that can accept large file such as Hex Editor Neo to edit the VMDK file and change the UUID. Look for ddb.uuid:

With the expanded hard disk, I installed the latest xCode, iOS SDK and various other apps just fine. Since the default Preview app does not work properly for graphics file due to the lack of QE/CI support, I replaced it with JustLooking, a freeware mac preview app that does not require QE/CI. Taking screenshots is done via VmWare's Capture Screen menu, or via Ctrl-Alt-PrintScreen.

However, everything was working great and I was developing my iOS app on the simulator when I encountered a kernel panic on the virtual Mac, causing VmWare to shut it down:

It took a while for me to realize that it wasn't the Mac that crashed first - it was the iOS app crashing on the simulator and taking the entire system down with it. Yes, any kind of crash, even one as simple as calling an unrecognized selector on an object:

-(BOOL)application:(UIApplication*)application didFinishLaunchingWithOptions:(NSDictionary*)launchOptions {   
[self dummyMethod];
    return YES;
}

If you are unlucky enough and the system is writing to the disk when the crash happens, the disk image will be corrupted and the Mac won't reboot to the desktop successfully. It will also be a tough task to recover any data from the corrupted disk image. Sort of funny but true. I was unable to determine the root cause and can only assume that it's due to hardware virtualization. The iPhone simulator architecture is actually x86, which suggests it is sharing the same CPU as the Mac, and any exception caused by the simulator will be escalated to the Mac if not handled properly.

To work around this, I put all my projects in a VmWare shared folder that is linked to a folder the host's hard drive so that the projects are still intact even if the Mac drive is corrupted. I also save working snapshots of the system that I can revert to in case something goes wrong. Most importantly, I add an exception breakpoint so that xCode will stop before an exception occurs, instead of letting it happen and crash the system:

My 10.6.8 upgrade is now completed and my next attempt is to upgrade to Mac OS X Lion (10.7.0), which will be another article. Meanwhile, I can now happily develop iOS apps on my laptop without purchasing a Mac.

↧

"Scanner Error" on Xerox WorkCentre 3119

March 23, 2012, 9:58 am

≫ Next: Experimenting with Casio Portable Analog TVs from eBay

≪ Previous: Mac OS X on VmWare - Upgrading from 10.6.4 to 10.6.8

Well, if you get this error, or some other hardware-related errors such as PAPER JAM, DOOR OPEN and TONER ERROR out of nowhere, the first thing to do is to perform a restart and re-installation of paper tray and toner, which will help if the error is triggered by a stuck or dirty sensor or switch.

However, for my case, there was also a continuous "tick-tock" noise as the device was trying to power up. The LCD showed "Warming Up..." for a very long time before reporting "Scanner Error". This error prevented the machine from being used for all purposes, even for non-scanning related tasks such as printing!

The source of the noise was traced back to the belt that moves the scan head one pass across the scan glass plate as part of the initialization process. The belt moved but the scan head did not move properly and the lamp (as seen in the following photo) remained in the left position. The scanner attempted to continue moving the belt until timeout and reported "Scanner Error".

My first thought was that the scanhead is unsure of its home position. There are 2 methods for a scan head to know its home position:

1. using a sensor as an optical slit or a switch
2. read a white band (made with a special paint) deposited on the scanner glass that sits parked in the scanner.

There was nothing obvious on the glass plate or elsewhere that suggests a mechanical malfunction. Apart from this, I can only assume that the belt is out-of-alignment and will need to be reseated, perhaps by following the service instructions found in the Xerox WorkCentre 3119 manual (downloadable here).

For now, I managed to bypass the error and allow the machine to be use for printing. I simply performed a CLEAR SETTINGS in the machine menu and rebooted the device. The device still attempted to initialize the scanner by illuminating the lamp but somehow did not attempt moving the scan head and will not stop at SCANNER ERROR. The error would however come back when photocopying/scanning is attempted.

I guess it's time for me to purchase a new machine.

↧

Experimenting with Casio Portable Analog TVs from eBay

April 11, 2012, 7:14 am

≫ Next: Prolific USB-to-Serial driver on Windows 7

≪ Previous: "Scanner Error" on Xerox WorkCentre 3119

Recently I won an eBay Auction for 2 portable TVs, the EV-570P and the TV-880D. Although both are analog TVs with a 3" LCD screen, the EV-570P supports both VHF and UHF while the TV-880D supports UHF only, as indicated in the channel ruler below the LCD:

Both sets have an automatic tuner and there is no option for manual tuning. Pressing the tuning buttons (next, previous) will scan and switch to the next available channel. A vertical scrolling bar (red for VHF, green for UHF) will be displayed showing the current channel (which is what the channel ruler is for).

I was disappointed to find out that although both TV sets are able to tune into Singapore channels and display the picture properly, only the EV-570P is able to play audio while the TV-880D simply plays white noises. I remembered the auction description clearly stated that both sets support PAL, which is the analog TV standard in Singapore. So what is the problem? I soon learned my lesson by looking at the back of both sets and by reading this Wikipedia article about different PAL standards:

The EV-570P supports PAL B/G/H/I while the TV-880D supports PAL I only. And Singapore is using PAL B/G standards. The following table from the above Wikipedia article explains why the TV-880D could not play the audio properly:

While PAL B/G and PAL I are color-compatible, the audio is not as it is transmitted on a different subcarrier frequency (5.5 MHz for PAL B/G and 6MHz for PAL I). The set was attempting to play the audio from a non-existent frequency, resulting in white noise. Notice that while this difference prevents the TV-880D from receiving Singapore channels in PAL B/G, the composite input is unaffected and can be used with other portable devices that have composite output such as VCD or DVD players

With some free time, I decided to disassemble the TV-880D to see if there is any hope of modifying it to support PAL B/G:

I quickly identified a through-hole crystal (highlighted in RED). However, with my oscilloscope, the frequency measured is 4.43MHz, the PAL color subcarrier frequency, not the sound subcarrier frequency.

My next attempt is to search for the service manual of the unit, which may offer some insights. I found manuals of other similar models, the TV-770B (for NTSC only), TV-770C/D/G (for PAL B/G/H, PAL I, PAL M respectively). The block diagrams of these models are quite similar and I can almost match it to my TV-880D:

Further studying the part list, I learned that the only major difference between the PAL B/G/H/I/M or NTSC set is the tuner (highlighted in green):

For TV-770B:

For TV-770C/D/G:

In other words, all that I was hoping to modify, the sound subcarrier frequency and the tuning range (VHF or UHF), is inside that "black box", the tuner. One of the replies to my post on this forum suggests that it may be possible to disassemble the tuner, change the crystal filters used for the audio subcarrier frequency and re-assemble it. However, as I found no elegant way to do that without break-opening the tuner metal can, I reassembled the unit unmodified and simply used it with my home circuit transmitting in PAL I.

The service manuals of the TV-770 and other Casio portable TVs can be downloaded here. I hope this will help others with similar problems.

See also:
Casio Pocket TV

↧

Prolific USB-to-Serial driver on Windows 7

May 24, 2012, 9:28 pm

≫ Next: Unlocking the Linksys SPA2102 Phone Adapter

≪ Previous: Experimenting with Casio Portable Analog TVs from eBay

My Prolific USB-to-Serial converter has been working well on Windows XP but refuses to work on Windows 7 64-bit. Windows first complained that the driver is unsigned:

and after that complained that the driver cannot be installed:

The root cause is clear - the Prolific USB-to-Serial driver is not compatible with Windows 7 because it is unsigned. Despite me choosing to install the driver anyway, Windows still refused to install it. After half a day trying various suggestions, I finally came up with a solution

1. Go to the registry branch HKLM\SOFTWARE\Policies\Microsoft\Windows NT\Driver Signing\BehaviorOnFailedVerify and set value as DWORD 0. This prevents Windows from rejecting unsigned drivers.

2. Open a command prompt and perform the following commands to enable Windows Test Mode, make use to use "Run as administrator":

bcdedit.exe -set loadoptions DDISABLE_INTEGRITY_CHECKS
bcdedit.exe -set TESTSIGNINGON

3. Use this tool to sign the driver and allow it to be used when Windows is in test mode. This applies to all DLL and SYS files in the driver package.

4. Reboot your machine and Windows should report that it is running in Test Mode at the bottom right corner:

5. Proceed to install the driver as per normal. Windows should accept the driver and install successfully:

The Prolific USB-to-serial driver can also be downloaded here, in case you're looking for it.

↧

Unlocking the Linksys SPA2102 Phone Adapter

June 7, 2012, 9:32 pm

≫ Next: Fixing "The CPU has been disabled by the guest operating system" when running OS X Lion on VMWare

≪ Previous: Prolific USB-to-Serial driver on Windows 7

I got myself a free Linksys SPA2102 SIP-to-PSTN adapter from a friend. However, to my disappointment, it is locked to iTalkBB and cannot be used with other providers. I searched for various methods and shared some of my finding here.

First, the SPA2102 supports both lines, namely Line 1 and Line 2. Some operators only lock Line 1, leaving Line 2 configuration free to change. If that is the case for your device and you only need to use a single line, simply use Line 2.

If that is not the case, enter the configuration menu (****) and try a user reset (877778#). Only do a factory reset (73738#) at a last resort! A user resets will only clear the SIP account settings whereas a factory reset will clear everything, including the web portal access passwords and set them back to manufacturer defaults. For an unlocked device, this will usually be either empty or easy-to-remember password (e.g. 0000). However, for a locked unit, this is a random number set by the SIP provider that locks the device on a per-unit basis. If you factory reset and gets your device to this state, you may as well buy a new one.

Next, try to enable the web portal access by 723646# and enter password 78778839#. After that, connect the WAN port to router and access Http://[ip address]:1980. Login with the following username/password pairs: user/1234 or admin/58724687.

If the password is rejected, you'll need to find our what the password is. Luckily, the locked device will keep on posting to the settings provisioning server looking for updated SIP server settings, using the username/password in clear text. You need to use a network capture tool such as WireShack, and look for the following data packet from the ATA:

<Admin_Passwd ua="na">58724687</Admin_Passwd>

When you have successfully opened the Admin portal, remove all items from the Restricted Access Domains list. You should be able to configure both lines.

If none of the above helps, try to do a firmware update. There are 2 ways, one is by following this guide which basically asks you to open a URL on the device web portal, pass in the firmware URL and wait for flashing to be done. Another way is to flash via a Windows tool, see this.

↧

Fixing "The CPU has been disabled by the guest operating system" when running OS X Lion on VMWare

July 26, 2012, 7:06 pm

≫ Next: Testing V9999 dual SIM card & dual camera quad-band phone from eBay

≪ Previous: Unlocking the Linksys SPA2102 Phone Adapter

My OS X Lion VMware image, which has always been working fine on my computer, a 2.53GHz Intel Core i5 HP notebook, stopped working when I migrated to a Dell Core i7 notebook. The machine would immediately stop at boot-up with the following VMware error message:

"The CPU has been disabled by the guest operating system. Power off or reset the virtual machine."

My first thought was that the VMware patcher to enable OS X guest on Windows was somehow not executed successfully. I tried to reinstall VMware, rerun the patcher, and install different versions of VMware many times but to no avail. I finally tried to enable verbose boot using:

sudo nvram boot-args="-v"

and the screen showed a little bit of more information, although not much useful:

There is no problem indicated by the above messages - it is simply part of the OS X boot process. The CPU was apparently halted as soon as the boot device was detected (indicated by the info 'root device uuid is...') without even attempting to boot further. Following a strange idea, I reverted the virtual machine to one of the previous snapshot, saved when OS X Lion was already running, and received the following warning about processor features difference:

The features supported by the processor(s) in this machine are different from the features supported by the processor(s) in the machine on which the snapshot was saved.

I chose to proceed, and guess what, my OS X booted up and functioned properly without further problems. For more than 2 weeks, due to time constraints, I simply worked with the virtual machine in that saved state, suspended it when done and reverting back to the snapshot (after saving all data) when I needed to reboot!

Recently I had some free time and determined to solve the problem completely. Following a Google search of the root device uuid message, I found this blog which describes a similar problem and the solution for VMware ESXi by editing the CPU ID:

- edit the VM settings with the VM powered off
- click on the Options tab
- click on the CPUID Mask menu item
- click Advanced
- scroll to the bottom of the window on the Virtual Machine Default tab
- under “Level 1″ set “eax” to 0000:0000:0000:0001:0000:0110:1010:0101. 

I am using VMware workstation, which does not have such an option to mask the CPU ID. However, using instructions from this forum, I was able to change the CPU ID by modifying the VMX file and add the following line:

cpuid.1.eax = "0000:0000:0000:0001:0000:0110:1010:0101"

With this change, my OS X Lion booted up properly and there is no need to work with it in a saved state anymore.

But what makes OS X Lion dislike my new laptop's processor and refuse to boot up? The fact that the unmodified virtual machine can still work on the new processor with the previously saved state implies that the processor difference is probably not critical - it was most likely a check that disables the processor if an unwelcome processor is found! With further research I realized that my new laptop processor architecture is Ivy Bridge, as compared with Sandy Bridge in my old laptop. Max OS X Lion doesn't like Ivy Bridge architecture, and thus refuses to boot up unless the CPU ID is masked to make OS X think it's running on a Sandy Bridge processor. What a hassle created by Apple, the virtual machine would have booted up properly and saved me precious time troubleshooting without this check.

Luckily, according to my research, this is probably not needed with Mountain Lion, since it supports Ivy bridge natively. I have not had the time to verify this, however.

On a side note, when trying to connect your USB device to the Mac virtual machine via VMware menu, you may receive the following error message:

"The connection for the USB device was unsuccessful. The device is currently in use"

When this happens, first check if the error message is telling you the truth. Stop all Windows applications that are currently using the device, reboot your laptop and the virtual machine and try again. If that doesn't help and you're using a new laptop with USB 3.0 ports, most likely you've hit a known problem - VMware does not support most USB 3.0 chips on Windows host. The Linux version does, however. This means, on Windows, you must revert back to a USB 2.0 or USB 1.1 port to avoid this error. So much for backwards compatibility!

In my case, I was lucky. My laptop has 4 USB ports, 3xUSB 3.0 ports and 1xUSB 2.0 port. I plugged the device to the USB 2.0 port and VMware was able to connect the device successfully.

↧

Testing V9999 dual SIM card & dual camera quad-band phone from eBay

August 11, 2012, 8:44 pm

≫ Next: Marshmallow 7G Android 4.0 Wifi+3G Tablet

≪ Previous: Fixing "The CPU has been disabled by the guest operating system" when running OS X Lion on VMWare

I got myself a cheap quad-band phone with dual SIM card, dual camera, USB port, and QWERTY keyboard, among other things, for a cheap price from eBay:

This phone apparently has many features, according to the seller:

Band        GSM 850/900/1800/1900MHz
Display Size 2.2 Inch
Resolution 320 x 240 pixels
Screen Color 260K Colors
Audio        MP3/WAV/AMR/AWB
Video        3GP/MPEG4
Image       JPEG/BMP/GIF/PNG/GIF
E-book      TXT/CHM/DOC/HTML
FM Radio    Yes, can play radio without earphone
Card         SD card up to 8GB
Data         WAP/GPRS
Camera   0.3MP
TV           Yes, analog TV with antenna
JAVA        Yes
Bluetooth    2.0

I spent a day playing with the phone, just for the sake of curiosity, and share some of my findings below.

Overall

2G (GPRS) only, does not support 3G mode
Proprietary charger and headphones, small keys and slow keyboard
The SIM card insertion direction printed on the back of the phone is reversed! This made me waste half an hour figuring out why the phone couldn't detected my SIM card

Voice/SMS

Outgoing calls/SMS are prohibited (incoming calls are still fine) until the SIM card is enabled by setting Settings > Dual SIM Switch > Use Default Mode and Settings > Dual SIM Settings > Dual SIM Open
It is a pain on this phone to send an SMS. The keys are tiny, too many menu options and too many pressed to switch between alphabet, symbols and numeric mode
Making and receiving voice calls seems fine during my testing

User Interface

New time zone prompt keeps on popping up. To turn off, go to Settings > Phone Settings > Auto update of date/time and set to Off
To enable key lock, go to Settings > Security Settings > Auto keypad lock. To unlock, press the BACK key (RSK) and #
Keypad tone default is ON. To turn off, go to User profiles > general > customize > Tone Settings > Keypad > Off

Data Connection

To configure GPRS, go to Services > Data Account > Add your GPRS APN here and Services > Internet Services > Settings > Profiles. Delete one or more existing accounts and add your GPRS profile pointing to the above data account
"GPRS/CSD connection exists" error will be shown if attempting to use camera when there is an Internet connection.
Many attractive options such as Entertainment News, Amazing Beauties, HD Movie, Ring Settings are unusable and report "No Services"

TV/Radio

TV works ok, although reception is a bit weak, but able to view most channels. Works without headset.
FM reception is a bit weak but otherwise work. Works without headset as well.
Consumes a lot of battery and lasts less than 1 hour with TV on.

Multimedia

Camera maximum resolution is VGA (640x480). Default resolution is QVGA. "Front camera" is actually on top!
Default Audio Player/Video Player reports "No Service", however File Manager can play some videos and music files (MP3, WAV)
Many video files can be played. Tested with 3GP and MP4 files. To play in full-screen mode press the A key.
DOC/CHM/HTML files are not supported as claimed. Only text files (TXT) are supported.

Browser

Default browser is very limited. Common pages, including built in facebook app default to Chinese versions with no font!
Opera Mini 3.3 is a good placement for the built-in browser
Opera Mini 6.5 crashes after showing splash screen, presumably due to low memory. Opera Mini 4.4 installs and works ok but is a bit slow. Only Opera Mini 3.3 Advanced Edition works well on this phone.

J2ME Apps

J2ME works in general. Installation can be from JAR files to either phone memory or storage card. Unicode is supported. Games are playable, albeit with tiny keyboard. Comes with some default games. Java apps and settings can be found in Fun & Games. Apps that consume lot of memory will hang at loading. Java apps launch in exclusive single task mode. Pressing the RED (hangup) key will terminate the app and go to home screen
Some apps such ABCMsn terminate upon connecting to network due to device prompts about airtime usage and the app cannot wait.
Some apps such as eBuddy can't work, the app will run and say "your phone does not have enough memory"

Bluetooth

Supports sending and receiving file via Bluetooth. Allows browsing remote Bluetooth folder as well.
Supports Bluetooth Audio service. To use it, user must start the connection from the device via Bluetooth >Search Audio Devices, otherwise connecting from Windows will say "Connection Failed". If successful, "Bluetooth Audio" will appear in Windows as a playback device. However, playing music on device still plays on loudspeaker, and attempting to play through "Bluetooth Audio" will cause the headset to be disconnected.
Support Bluetooth Serial Ports and act as GSM modem. Tested with Hyperterminal. Flow control must set to NONE for it to work. The modem is able to send SMS but cannot call out. "NO CARRIER" is reported if attempted, potentially because Bluetooth and phone share the same radio and can't be simultaneously on

USB Connection

When connected via USB, the phone supports many mode, including USB charging.
Support USB mass storage mode for both internal devices & storage card
Support USB Webcam mode and works well with Webcam Viewer, up to VGA quality
COM port mode is included in the list of modes, however Windows 7 64-bit never finds drivers for it and I am not sure what this is for.
Java Connection is also included, however Windows 7 plays the "device connect fails" sound and I am also not sure what this is for. Perhaps to debug Java apps?

After finishing testing the phone, I put it in my drawer and did not use it again until a few months later when my main phone failed. This is when problem began to appear. The battery capacity is poor and lasts for less than a day even with little usage. Occasionally the phone would just hang and require a reboot. I complain to the seller and although he was nice enough to send me a new battery, my usage of the phone stopped a few weeks later when it failed to boot up and hangs at the charging indicator.

In short, just as many other cheap Chinese phones from eBay, this is good as a toy and for experimenting, but never as a primary phone. For that purpose, my old Nokia 3210 is far better and can still serve me well after all these years.

↧

Marshmallow 7G Android 4.0 Wifi+3G Tablet

September 14, 2012, 4:31 am

≫ Next: Using SOAP web service in an iOS application

≪ Previous: Testing V9999 dual SIM card & dual camera quad-band phone from eBay

I recently purchased the Marshmallow 7G, an Android 4.0 tablet that has Wifi and 3G, supports USB hosts with HDMI output, and other features.The Marshmallow 7G is distributed through w3gear, a Singapore company, and is available for purchase online via w3gear's official website at a price of 289 SGD at the time of writing this article (Sept 2012).

The tablet has a home button, side volume buttons and a front camera:

On first look, this Android tablet is cheaper than the Dreambook W7, which I have mentioned in details in a previous article, and is also much lighter (345g compared with 500g or more for the Dreambook) and thinner. I decided to perform a thorough test of the tablet and share my finding here, which I hope will help others who intend to buy the tablet.

Overall

The default screen orientation is landscape and an accelerometer will rotate the screen to portrait appropriately. However, should you decide to turn off auto-rotation, the screen will default back to Landscape and there is no options to set to Portrait mode.

Same as the Dreambook W7, there is no restart option on this device. You can only turn the device off and turn it back on.

Call & SMS

As a 3G Android tablet, the device has the Android default Dialer and Messaging applications. Although making or receiving calls as well as sending/receiving SMS works fine, the default dialer has problems dialing USSD code. For example, if you are on a prepaid card and dial your network provider's USSD code of *100# to check account balance, the dialer would hang forever at the "Dialing" screen. No 3rd party dialer I tried would help, since the problem seems to be in the radio firmware

The loudspeaker volume is also too soft even at maximum output. There is no headset option and calls are made on loudspeaker.

Internet Access

The device supports 3G, which works fine with my SIM card. However, it takes up to 5 minutes upon a cold boot for the device to lock on to a network provider. Once registered onto a network, 3G reception seems stable.

The default browser user agent defaults to a PC browser, which will just make loading web page slower. To overcome this, I use Opera Mobile.

Tethering works well in all my tests. Wifi reception is reasonable and the device connects to my home router with no issues.

Location Services

There is neither GPS support nor a compass on this device. Approximate location via Wifi/CellID is also not available, even though the device supports 3G and should, in theory, know its approximate location via the cell network. The Nook Tablet, however, can identify its approximate location as long as it is connected to a Wifi network.

Bluetooth & Peripherals

In my test, I am able to send a text file from a phone to the PC via Bluetooth. However, the device is unable to receive a file via Bluetooth - the PC reports timeout despite numerous attempts to tweak the Bluetooth settings. Playing audio via Bluetooth works fine, however.

USB host, one of the feature that convinced me to buy this tablet, works well. Mouse, keyboard and FAT16/FAT32 thumbdrives are supported, either independently or via a USB hub. The device comes with a proper adapter which would allow you to connect your USB devices to the micro-USB port.

I did not test the HDMI output. To stand any chance of having the Android screen projected on your LCD TV, however, you would need to have an appropriate HDMI (Female) to mini-HDMI (Male) converter.

Battery

Most of my tests with the device have been positive so far and I was intending to use the tablet for daily needs when I realized that the battery level was decreasing very fast. With 3G on, in standby mode with no major application running, the tablet battery is dead within a day or so. Try to play a video with both the phone radio and Wifi off, using your headset, 15% screen brightness, full battery and the battery is dead within 2.5 hrs. Although I can understand the battery drain while playing video, I do not understand why the device consumes so much battery even in standby mode with no other applications.

All this indicates poor battery capacity, which I can tell from the weight of the phone. It should have been heavier to support a higher capacity battery.

The verdict

Giventhat I explored the Dreambook W7 a year ago and was pretty disappointed, I was pretty surprised when testing the Marshmallow 7G tablet for the first time - almost everything worked as described. However, the excitement did not last long when I realized that the poor battery is a show-stopper for this device - the short battery life would not allow you to use it for anything other than experimenting. On that basis, I resold the device and purchased a Nook Tablet, which although does not feature 3G, can still serve me well due to its excellent battery life - a month on standby and up to 8 hrs of continuous usage.

↧

Using SOAP web service in an iOS application

October 18, 2012, 5:23 am

≫ Next: Using ApnsPHP library with CodeIgniter to send Push Notifications to iOS devices

≪ Previous: Marshmallow 7G Android 4.0 Wifi+3G Tablet

I recently needed to connect to SOAP web services written in .NET/PHP from one of my iOS applications when I realized that is there is no built in support for SOAP messages (although iOS5 and above has support for JSON via the NSJSONSerialization class). Obviously I could decide to construct the SOAP XML messages manually and POST them to the web service via NSURLRequest, I soon realized that this approach is not feasible due to the many (50+) web services that my application needs to use.

Fortunately I find wsdl2objc, an Objective-C code-generator for SOAP web services which automatically parses the web service WSDL definition and generates the appropriate classes to send a request and parse the response.

The tool is a simple Mac OS app that asks you for the location to your web service WSDL definition, which can be a URL or a local file, and where to put the generated code:

Supply the required information and press Parse WSDL. The tool should quickly finish and give you the resulting source code files, which look similar to below:

Among the files, MyService.h and MyService.m contain the definition and implementation of your web service based on the WSDL file. The rest of the files are supporting classes to parse the XML messages. If you have more than one WSDL file, run the generator for each WSDL and include all the generated service class files in your project. The supporting classes only need to be included once.

You should have no issues compiling the generated code and make a simple web service call using the sample code given on the wsdl2objc home page. However, depending on the web service, you may run into any of the following problems:

1. The web service does not receive the parameters passed in properly, or, for if the service is written in Java, exception "Cannot find dispatch method for {serviceURL}/{serviceName}" will be thrown on the server.
This is because the namespace attribute, xmlns, is missing from the SOAP body envelope. To fix this, locate the method:

- (NSString *)serializedFormUsingHeaderElements:(NSDictionary *)headerElements bodyElements:(NSDictionary *)bodyElements

Look for the following block of code

if((headerElements != nil) && ([headerElements count] > 0)) {
        xmlNodePtr headerNode = xmlNewDocNode(doc, soapEnvelopeNs, (const xmlChar*)"Header", NULL);
        xmlAddChild(root, headerNode);

        for(NSString *key in [headerElements allKeys]) {
            id header = [headerElements objectForKey:key];
            xmlAddChild(headerNode, [header xmlNodeForDoc:doc elementName:key]);
        }
    }

and add the following code just below the above code:

xmlNewNs(root, (const xmlChar*)"http://localhost/MyService", (const xmlChar*)"ns1");

Remember to modify the URL to point to the correct namespace of your web service. ns1 in the above code specifies the namespace that each of the web service method must adhere to. Some web services may not require this, but for those that require, failing to provide may result in issue (2) below.

2. Simple values (integers, strings) are received property by the web service, but complex values (custom types) are received as blank
This is because the web service methods do not specify the namespace (ns1) it belongs to. Fixing this requires manual modification of MyService.m for each of the web service in the WSDL file. Assuming your service is called MyService, look for the following line:

NSMutableDictionary *bodyElements = nil;
bodyElements = [NSMutableDictionary dictionary];
if(callerParams != nil) [bodyElements setObject:callerParams forKey:@"MyService"];

Replace the last line with

if(callerParams != nil) [bodyElements setObject:callerParams forKey:@"ns1:MyService"];

3. Special characters such as ampersand will not be received on the server
This is because the generated code forgets to escape the ampersand character, as required by the SOAP protocol. To fix this, open USAdditions.m, locate the xmlNodeForDoc method, and replace the method code with the following:

There is a reason why the last block of code is a colorful image :). Had it been text, Google Blogger will try to unescape the ampersand character (the withString part), resulting in wrong code.

With all the above changes, the generated code should be compatible with most SOAP web services.

↧

Using ApnsPHP library with CodeIgniter to send Push Notifications to iOS devices

November 23, 2012, 9:09 pm

≫ Next: IBM Thinkpad 600 Memory Upgrade

≪ Previous: Using SOAP web service in an iOS application

I recently needed to use PHP to send Push Notification to iOS and Android devices. While sending push notification to an Android device is straight forward once the device registration ID is known (just a few cURL calls to POST to Google API server), it was a mess trying to do the same for iOS devices.

First, Apple likes to make things complicated for developers and the sending of push notification to iOS devices requires the use of a PEM key, or two keys to be exact (one for debug and one for release), in order to communicate with the server. To implement this communication protocol, the usage of a third party library such as ApnsPHP is required and the code to send a push message is as follows:


// Using Autoload all classes are loaded on-demand
require_once'ApnsPHP/Autoload.php';

$push=new ApnsPHP_Push(
ApnsPHP_Abstract::ENVIRONMENT_SANDBOX,
'server_cerificates_bundle_sandbox.pem'
);
$push->setRootCertificationAuthority('entrust_root_certification_authority.pem');
$push->connect();
$message=new ApnsPHP_Message('1e82db91c7ceddd72bf33d74ae052ac9c84a065b35148ac401388843106a7485');
$message->setCustomIdentifier("Message-Badge-3");
$message->setBadge(3);
$message->setText('Hello APNs-enabled device!');
$message->setSound();
$message->setCustomProperty('acme2',array('bang','whiz'));
$message->setCustomProperty('acme3',array('bing','bong'));
$message->setExpiry(30);
$push->add($message);
$push->send();
$push->disconnect();

However, while I got the code working on first try, subsequently I encountered a problem while trying to use the code from a CodeIgniter controller:

Fatal error: Uncaught exception 'Exception' with message 'Class file '/wwwdata/www/html/application/controllers/CI/DB/mysql/result.php' does not exists' in /wwwdata/www/html/application/controllers/ApnsPHP/Autoload.php:49

A Google search of the error message found only a few results. Among them, this stackoverflow post proposed that the Autoload feature of PHP, used in the APNS library to avoid the need of manually declaring require_once every time a new PHP class is needed, is conflicting with CodeIgnter, which perhaps is also trying to use the same Autoload feature to initialize its MVC model. Another post here described the same problem but provided no solution.

After some consideration I decided to try the hard way - which is to remove the use of Autoload and replace it which manual require_once declarations. With this modification, the require_once declaration above is changed to the following long list:

require_once'ApnsPHP/Abstract.php';
require_once'ApnsPHP/Exception.php';
require_once'ApnsPHP/Feedback.php';
require_once'ApnsPHP/Message.php';
require_once'ApnsPHP/Log/Embedded.php';
require_once'ApnsPHP/Log/Interface.php';
require_once'ApnsPHP/Message/Custom.php';
require_once'ApnsPHP/Message/Exception.php';
require_once'ApnsPHP/Push/Exception.php';
require_once'ApnsPHP/Push/Server.php';
require_once'ApnsPHP/Push/Server/Exception.php';

Surprisingly, only with some extra manual modifications of a few classes in the ApnsPHP library to include the necessary classes, the library works and is able to send push notification from CodeIgniter. For example, the following line must be added at the top of server.php, since it refers to Push.php:

require_oncerealpath(dirname(__FILE__).'/..').'/Push.php';

The complete library with the above modification can be downloaded here. As I said, this is not an elegant solution due to too many manual modifications required. Feel free to comment if you have any ideas for a better approach which does not involve extensive modification of the library.

↧

IBM Thinkpad 600 Memory Upgrade

December 18, 2012, 11:04 pm

≫ Next: Using 3CX Call Control API in a .NET application

≪ Previous: Using ApnsPHP library with CodeIgniter to send Push Notifications to iOS devices

I got an IBM Thinkpad 600 from eBay that has 32MB RAM and a working CD drive with no hard drive or external floppy disk drive and tried to see what it can be used for.

With 32MB RAM and a 20GB IDE hard drive installed, operating systems such as MS-DOS, Windows 95 and Windows 98 work well on this laptop. The three Linux variants that I tried (Slax, Puppy, DSL) manage to boot up to the shell but none could actually boot up to a usable GUI such as KDE or X-Window, presumably due to limited memory. Puppy Linux displayed a distorted and unrecognizable desktop window once it finished booting whereas DSL shows a working desktop with a very pale color theme and hence, unusable. Even Hiren Boot CD 8.4 fails to boot up due to not enough memory to create the RAM drive.

As my intention is to get Linux running on this laptop, I proceed to upgrade the RAM for this notebook. The official Thinkpad specs specified that the maximum RAM allowed is 288 MB with 2x128MB SO-DIMM 144-PIN RAM modules installed in the two available slots, plus the 32MB onboard memory.

At first I decided to purchase two MT4LSDT1664LHG-133C1 128MB 133 MHz CL3 memory modules from eBay for a cheap price. Once installed, the laptop detected correctly that 288MB has been installed but could not boot up. The system hangs after the memory counts and there is no POST beep. This turned out to be the memory speed - the onboard memory module is PC66 and hence will not work with PC133 modules. I therefore bought another two MT8LSDT1664HG-10EB1 100MHz CL2 memory modules and decided to try again.

This time the system boots up successfully and seems to have no problems with Mini Windows 98 from Hiren Boot CD. However, the detected total memory is 224 MB, not 288 MB as expected. A mysterious amount of 64MB has been lost. With further experiment, I realized that one of the memory slots seems to recognize both of my 128MB memory modules as 64MB only while the other slot detects their correct capacity, which is very strange.

Booting up into EasySetup by pressing and holding the F1 key while turning the machine on in order to checks the memory count, I noticed the following:

The above text reads "Installed: 294912 KB/Usable: 228800KB" which means the system detects the correct memory module size, just that it is unable to use the extra 64MB. Why? The answer is provided in this forum discussion, post #4.

In short, the Thinkpad 600 has 2 physical memory slots and supports 4 memory banks (banks 0, 1, 2, 3) in total. Memory slot 1 is assigned 2 memory banks (bank 1 and bank 3). However, memory slot 0 is assigned one memory bank (bank 2) since bank 0 has been taken by the onboard 32MB memory module.

The 128MB PC100 module reports itself as having 2 memory banks of 64MB each. When the module is inserted into slot 1, everything works well because both memory banks are available. When it is inserted into slot 0 that has only one memory bank, 64MB is lost because the other memory bank cannot be used. This explains why the total amount of usable memory is 128+128+32-64 = 228MB.

In order to achieve the maximum possible 288MB, I would have to find a 128MB memory module which reports itself as having a single bank of 128MB. The difference seems to be speed-related. For example, the 128MB PC133 I purchased earlier, albeit not compatible, seems to report a single 128MB memory bank and hence, helps achieve the maximum 288MB memory. The other hint that I have is the number of memory chips on the memory module - the PC133 module has 2 whereas the PC100 has 4.

In the end, because there is no need for an extra 64MB of RAM on this laptop, I decided to stay with 228MB.

↧

Using 3CX Call Control API in a .NET application

January 24, 2013, 9:18 pm

≫ Next: 3CX Caller Data Record (CDR) output file format

≪ Previous: IBM Thinkpad 600 Memory Upgrade

In one of the projects at work I attempted to use the 3CX Call Control API (see this) from my .NET application and encountered unique challenges because the Call Control API is only available from a .NET application running on the same server as the 3CX machine. This means, even if the sample .NET application to demonstrate the API provided by 3CX is working well, it is of little usefulness if you want to expose the API to your custom application which undoubtedly must be running from the client machine and not on the same server.

My design is to write an API wrapper that runs on the 3CX server, receives client requests via HTTP, interacts with the 3CX Call Control API to perform the necessary actions and returns the API response back to the client also via HTTP.

With this design, the first attempt is by using an ASP.NET SOAP web service, which unfortunately does not work. First, the web service fails to start due to a BadImageFormatException once the API DLL 3cxpscomcpp2.dll, is added as a reference to the project

System.BadImageFormatException: Could not load file or assembly '3cxpscomcpp2.dll' or one of its dependencies. An attempt was made to load a program with an incorrect format

Knowing that this is because of the format of the DLL (32-bit vs. 64-bit) and the architecture of the project (x86 or x64), I tried to changed the project platform but neither x86 or x64 works. I also changed the application pool settings inside IIS following this article, which also does not help.

Next I noticed that the API DLL is a 64-bit DLL and attempted to install 3CX on a 32-bit machine to retrieve the 32-bit version of the DLL. This time, the error message when loading the web service changed:

Could not load file or assembly '3cxpscomcpp2' or one of its dependencies. Modules which are not in the manifest were streamed in. (Exception from HRESULT: 0x80131043)

The error message is not very useful and several Google searches did not return any working solutions. This has to do with the fact that the 3cxpscomcpp2.dll uses a native C++ dll named sl.dll. For the API to initialize properly, both DLLs are required. Despite substantial research, I could not find any reasons why sl.dll fails to be loaded and thus giving up integrating the DLL with ASP.NET web service.

My next attempt is to use a Windows Communication Foundation (or WCF) aplication instead, and not ASP.NET web service. The application will hook up to a HTTP port on the machine, listening to HTTP request via POST/GET and returning the response in JSON. This time, the application has no difficulties connecting to the API and things work as expected.

The next challenge is encountered when one of the developers using my APIs reported a strange error when calling my API from jQuery:

Origin http://localhost:8888 is not allowed by Access-Control-Allow-Origin.

To fix the error, one of the following must be done:

The browser must allow cross domain calls. For Chrome, you can launch chrome with the parameters --disable-web-security and cross domain calls will be allowed.
The server response must contain the following header to allow calls from any origin:
Access-Control-Allow-Origin:*
The server sends the response in JSONP

Because (1) is out of the question since the calling web site must be able to support different browsers while (3) is not possible yet from WCF, I have chosen (2). Luckily I found the following MSDN blog which proposes a solution that does not require any code changes:

Import the WebHttpCors DLL provided by the author
Modified app.config to add the tag to allow cross-domain calls.

It works well and the API can be called from jQuery with no issues. However, during the project, I also noticed several limitations with the 3CX Call Control API:

There is no API to put a call on hold. The closest you can get is to transfer the call to a parked extension. To unhold the call, make a call to the parked number and you will be able to continue with the parked call.
There is no API to retrieve the call history and the application needs to manually parse the 3CX Call History database located at C:\ProgramData\3CX\Data\Logs\CallHistory. In this aspect, the approach of using a WCF application instead of ASP.NET poses a major advantage because as a Windows application, WCF has no difficulties accessing files located on different paths on the server. This is also needed to read the call recording files located at C:\ProgramData\3CX\Data\Recordings

I hope 3CX will be able to introduce more APIs in the future to solve the above mentioned limitations.

↧

3CX Caller Data Record (CDR) output file format

February 15, 2013, 9:18 am

≫ Next: Experimenting with the WTV020 voice playback module

≪ Previous: Using 3CX Call Control API in a .NET application

Although 3CX, the most common software PBX for Windows, comes with a few ways for user to generate detailed call reports and usage statistics in various different formats, many advanced users such as large corporation often find the reporting feature inadequate due to the need to generate custom reports which aren't supported by default. To do so one would need to retrieve the raw CDR data, either by accessing the call database directly or by analyzing the CDR text files generated by 3CX, and generate their own reports.

Direct access to the 3CX call database

The 3CX database, containing call records and various other PBX settings, in in PostgreSQL format and will be accessible via a PostgreSQL client such as pgAdmin. The authentication credentials can be retrieved from the file 3CXPhoneSystem.ini found in the C:\Program Files\3CX PhoneSystem\Bin folder.

Once connected to the server, the 3CX database is located at Servers>3CX>Databases>phonesystem>Schemas>public>tables. Call information is consolidated into 3 tables, namely calldetails, callhistory2, callhistory3. For general call history statistics, records from table calldetails would be sufficient.

Although not officially documented, various online resources describing the database format are available. Refer to this and this for more information on the database schema.

Interestingly, the credential provided in the 3CXPhoneSystem.ini cannot be used to access other tables in the database. I do not yet know how to access other tables.

This approach has a few drawbacks, among them the need to know the 3CX database password (which may be machine-specific), and the dependency on the database schema, which could change in future 3CX versions.

Analyzing the CDR text files

If you do not wish to connect to the database, another approach is to read the CDR text files that are generated by 3CX as calls are made. These files are found in the C:\ProgramData\3CX\Data\Logs\CallHistory folder. (And for those who are interested, the call recordings WAV files, if recording is enabled, are found in C:\ProgramData\3CX\Data\Recordings, with recording for each extension saved in a subfolder having the same name as the extension number)

The default format of the CDR output is quite straightforward. Each line in the log file is comma separated and will have at least 8 fields. For each call from the initiating to completion state, several lines will be written to the CDR log file as the call progresses. The description of the fields are below:

Field #0– State of the call. Possible value are Connecting = 1, CallEstablished = 2, PartyAdded = 3, PartyRemoved = 4, PartyChanged = 5, Disconnected = 6, DestNoAnswer = 7, DestIsBusy = 8, DestNotAvail = 9, RecordingSaved = 10
Field #1– The time of the call state change, in the format yyyymmddhhmmss.### where ### is the number of milliseconds
Field #2 – History ID of the call on the PBX
Field #3– Internal source number of the call
Field #4– Internal destination number of the call
Field #5– External source number of the call if the call originates from an external number, otherwise, same as Field #3
Field #6– External destination number of the call if the call terminates on an external number, otherwise, same as Field #4
Field #7– Type of call (1 = voice call, 0 = fax call)
Field #8 (Optional)– Any additional information about the call. This is often the name of the 3CX call queue if the call is involved in a queue.

For example the following line

1,20130414105548.819,00000BD538E62E50_2539,80001,10011,012345678,10011,1,"Customer Service Queue::*"

tells us that on 14 April 2013 at 10:55:48.819 (Field #1), the call having a history ID of 00000BD538E62E50_2539 (Field #2) was in the connecting (Field #0 = 1) state. The call was made from external number 012345678 (Field #5), reaches the 3CX digital receptionist on 80001 (Field #3), and was routed to extension 10011 (Field #4 = Field #6 = 10011). The call was a voice call (Field #7 = 1) on the Customer Service Queue (Field #8)

With knowledge of the file format, one would think that creating the call history from the CDR output would be an easy task. Unfortunately, this approach has a few challenges, with some being more critical then the rest:

The CDR output files are not updated immediately after a call is made, but rather, after a certain interval configurable from 3CX Admin Portal. My experience shows that even with the shortest possible interval set, some times the CDR output takes a while to be updated, resulting in outdated call history information.
With each call, a few status lines are written to the CDR as the call progresses, e.g. Connecting>CallEstablished>Disconnected. There could be more intermediate statuses if the call involes a transfer or is a conference call. Deducing the necessary information (e.g. call duration) could be tricky.
If an extension-to-extension call is made (e.g. inbound calls), for each status change, 2 almost similar records will be created for each extension involved - with only the extension number being different. Depends on the usage, it may be necessary to filter out such records after processing the CDR, which will slow down the performance of the code.
If a call reaches a queue and is diverted to the agents involved in a queue, CDR records will be created as 3CX tries to find available agents in the queue. This means, if a 10-agent queue has only 1 available agent at the time of the call, you will see 9 records of unsuccessful calls created for the attempts to reach the unavailable agents before the actual successful call record. Again, filtering out these records could be tricky.

There could be more problems as there are more CDR records from complicated scenarios which I have yet to encounter. However, in my case, with simple user requirements (knowing the total number of successful/failed calls, total duration, etc.), my usage of .NET LINQ to analyze the CDR so far seems adequate.

Since each of these approaches has its own problems, the best would be for 3CX to provide a method to retrieve the call history as part of the Call Control API, which is not yet possible as at 3CX version 12.

↧

Experimenting with the WTV020 voice playback module

March 20, 2013, 10:03 am

≫ Next: Ramsey TV6C Analog Television Transmitter Kit

≪ Previous: 3CX Caller Data Record (CDR) output file format

I recently bought a WTV020 module from eBay, advertised as 'MP3 VOICE SD CARD VOICE MODULE U-DISK AUDIO PLAYER WTV020-SD' in one of the ads, in the hope that it would allow me to develop my own SD-card MP3 player using a PIC18/PIC14 micro-controller. Unfortunately, I was in for a disappointment as soon as I read the datasheet...

The module works on 3.3V and has two operating modes. The first is a propietary serial mode where commands are sent via a 2-line (DATA, CLOCK) serial connection. And the second mode is referred to the datasheet as an 'MP3' mode, which allows the usage of the module for playback without the need of a micro-controller:

Basically this mode allows the module to be operated directly just by connecting signals to the volume, next/previous, play/pause pins. The module would look up the supported tracks (either WAV or AD4) on the SD card and play them in order. This mode makes the module look like an MP3 player (or so the seller thought), and therefore the item is advertised as an MP3 module! There is actually no support for playback of an MP3-formatted file, as mentioned in the item title. Should I consider this false advertising?

Anyway, after an initial disappoinment, I proceed with testing nevertheless and below are some of my observations:

Audio playback is via pulse width modulation (PWM), hence the quality is poor. Adequate for low quality voice, but definitely not for music.
Only small size SD cards (32MB, 64MB, 128MB) seem to work well. Cards with larger capacity (256MB and above) tend to work inconsistently or do not work at all.
Despite the schematic showing a direct connection with a speaker from the SPK+ and SPK- pins, you'll definitely need an amplifier such as LM386 (or a crystal earpiece), before you'll be able to hear anything.

I have not yet had the time to try the serial mode for playback on this module yet. Or perhaps my time should be better spent on figuring out how to get a free sample of the 'real' STA013 MP3 player IC from STMicroelectronics and get it to work. All of my attempts to submit a free samples order, however, have been rejected.

↧

Ramsey TV6C Analog Television Transmitter Kit

April 25, 2013, 3:41 am

≫ Next: Custom USB HID device using PIC18F4550

≪ Previous: Experimenting with the WTV020 voice playback module

Six months after I purchased a TV6C analog television transmitter kit from eBay, I finally got the motivation to open the package and attempted the construction! Yes, the idea of having sort of a 'real' working television transmitter in my room was great, but the amount of time and efforts that will need to be spent has prevented me from trying the kit previously.

This is an all-in-one kit that provides you with everything from the PCB, capacitors, resistors and inductors to even the plastic box for the transmitter unit. Of particular note is the IF can-style RF transformer for the audio sub-carrier frequency (4.5MHz for NTSC-M in the US), which would have been very tricky to purchase.

This is the packaging for the kit:

The circuit is a low-power TV transmitter with separate video and audio input, and a 2N3866 power amplifier stage to increase the transmission range:

This is the full circuit diagram:

A professional-looking PCB is already provided in the kit and except for a 220pF capacitor which has to be soldered underneath the board, the rest of the soldering is straightforward. This is the completed PCB after the soldering job:

My transmitter worked upon first power on. The following picture shows the circuit transmitting on Channel 4 using the output from an AKIRA DVD Player, with the signal being received on a Casio EV-550B portable television:

This is the front and the back side of the unit box. Note that the LED indicator was added by me and not part of the original kit:

During my testing the completed circuit works beautifully with clear pictures on my Casio portable television. The transmission range is approximately 100m with no obstacles in between. The only disadvantage of this unit is the 4.5MHz IF transformer designed for the NSTC-M TV system in the US. This means that you will still be able to feed in a PAL video signal with audio, but the receiving television may not be able to play the audio properly unless it's a smart enough to detect the TV system and audio subcarrier frequency separately. In my case, with the AKIRA player setup to output NTSC and the Casio EV-550B supporting NTSC-M, the setup works just nice.

In order to transmit PAL, you will need to find an IF transformer having the audio sub-carrier frequency of the PAL system supported by your television (5.5MHz, 6.0MHz or 6.5MHz). I am planning to scrap an old PAL television, find the part and try again :)

Another minor disadvantage of the unit is the single mono audio RCA input while most players nowadays output stereo. To overcome this, you will need a stereo to mono converter circuit:

The full manual for the kit can be downloaded here for those who are interested.

↧

Custom USB HID device using PIC18F4550

May 15, 2013, 4:44 am

≫ Next: Hyper-V and hardware assisted virtualization

≪ Previous: Ramsey TV6C Analog Television Transmitter Kit

With some free samples of PIC18F4550, the most popular micro-controller for hobbyist to build custom USB gadgets, several type of sensors, a PS2 keyboard and a Game Boy camera unit, I decided to build a custom USB device that would process input from these peripherals and display them on my computer. This article shows the final product that I came up with and shares some of my findings.

USB firmware for the PIC18F4550

Unlike UART which requires just a few lines of code to configure the baudrate, USB is a very complicated protocol and it would take some serious efforts just to get your computer recognize the PIC as a USB device.

The first step is to write the firmware for this PIC which would allow it to perform USB enumeration upon connected and necessary communication before it is usable by the host. The next step would be to write an application running on the host that communicates with the USB device and displays the relevant information. Fortunately, I do not have to develop the firmware code and the Windows software from scratch, as a working setup with code sample is provided in this website which includes:

A firmware for the PIC18F4550 which reports itself as a generic USB Human Interface Device (HID)
A .NET application written in C# that performs basic communication (e.g. toggling LEDs) with the PIC

The source code for the Windows application is developed in C# using Visual Studio and consists of 2 projects:

usbGenericHidCommunications_3_0_0_0: the library for low-level USB communications
WFF Generic HID Demo 3: the application communicating with the USB device using the above library

As a generic USB HID device, the PIC will wait for requests to be sent from the host and replies with a data packet in response to the command. Since each data packet is 64 bytes and the host cannot send more than 1 request per millisecond, the maximum throughput is 64KB/sec. Although that is well below even the maximum speed for USB 1.1 (12MBit/sec), it is more than enough for my purpose.

I got the sample code working upon first try:

Interfacing additional peripherals

To make it a more useful USB device, I connected the following peripherals to the PIC and modify the Windows software accordingly:

DHT11: temperature and humidity sensor using proprietary 1-wire protocol
DS1621: I2C digital thermometer by Maxim
LM35: analog temperature sensor by Texas Instruments.
DS1307: I2C real-time clock by Maxim
HCSR04: ultrasonic distance sensor
A3144: hall-effect sensor
A PIR sensor for motion detector

Using this AVR source code to interface a PS2 keyboard and this for the Game Boy camera and adapted them for use with the PIC, together with a Nokia 5110 LCD for displaying of status, it took me 1 week to complete my final circuit:

The LCD screen at a closer look, with the third line showing the temperature detected by the 3 sensors and the fifth line showing the distance detected by the ultrasonic sensor and the humidity detected by DHT11:

Extra C# code is added to the Windows software to cater for the added devices, for example, to query the temperature of the DS1621:

The PIC firmware is also modified to look out for the command 0x83 sent by the host and reply with the status of the DS1621. This change is done inside the processUsbCommands function:

The final Windows application to show the added peripheral status looks like the following:

Interfacing the PS2 keyboard

This is a bit challenging due to the need to watch out for the PS2 clock to detect when a key is pressed and read the key's scan code while at the same time processing any possible commands sent from the host. This is solved by using interrupt - connecting the CLK line of the keyboard to the INT0 pin and process the keyboard input in the highPriorityISRCode() function:

The key is then stored in a buffer, to be retrieved the next time the host queries the list of pressed keys. A text box is added to the host software to show which key has been pressed.

Interfacing the Game Boy camera

Codes are written to take a picture using the Game Boy camera and transfer it back to the host, via multiple data packets of 64 bytes each, to be displayed in a PictureBox in the .NET application. It takes around 5 seconds for a 16KB image to be taken and sent back to the host. So much for a USB connection!

The following is a picture of a calculator taken using this method. If you can see it, the calculator screen is showing 12345678.

Bearing in mind the limited capabilities of this camera, the following problems were noted:

The last 5 lines of the image does not contain data. This is because the actual resolution of the camera in my case is 128x123 only, despite the camera indicating a resolution of 128x128 during the communication protocol.
In some cases the image appears brighter than normal, possibly because of an increase due to the exposure time due to the waiting time between host commands.
Most importantly, the camera sensor experiences interference causing "dead" pixels which will create periodic horizontal lines as part of the bitmap.

Issue (3) is also encountered if the image data taken using the same code is transferred via UART or written to an SD card, instead of via USB. My post on this forum yields a few replies suggesting that the analog-to-digital converter of the PIC experiences interferences when the camera is capturing photo affecting the ground reference causing the change in pixel intensity. Despite trying various different methods such as making the ground connection large, changing the reference voltage for the ADC, or even correcting the intensity in software, I could not solve the issue and have to leave with the interferences in the captured photos.

However, needless to say, I have learned a lot about the USB protocol and acquire more experience interfacing various peripherals to the PIC during the project, The modified source code for both the firmware and the Windows application can be downloaded here.

↧

Hyper-V and hardware assisted virtualization

June 12, 2013, 7:40 am

≫ Next: Converting an Outlook Form Template (OFT) to a .NET Windows form in Visual Studio

≪ Previous: Custom USB HID device using PIC18F4550

During one of my experiments with a virtual machine in Virtual Box running on Windows 8, I notice that the Acceleration tab of the System settings is grayed out:

A quick Google search reveals that this would only happen if the host machine processor does not support hardware-assisted virtualization. This is obviously not the case for me, since I am on a late-2012 Mac Mini with a Intel Core-i5 3210M CPU, which supports the VT-x instructions. However, I verified this using Intel Processor Identification Utility, CPU-Z 1.66 and SecurAble, with all returning the same disappointing result:

I rebooted my machine, which has been set up to triple boot Mac OS, Windows and Ubuntu Linux, to Mac OS and execute MacCPUID. Surprisingly, the tool reports the exact opposite - my CPU seems to support VT-x instructions:

So which tool is telling the truth? I know for one thing, my machine runs the Windows Phone 8 Emulator just fine. And since this emulator makes use of Hyper-V, which requires the CPU to support hardware virtualization (and possibly SLAT as well), my CPU must be fully virtualization capable.

In a last attempt at this issue, I turned off Hyper-V by executing the following and reboot:

bcdedit /set hypervisorlaunchtype off

Guess what, after a reboot, all tools now reported that my CPU is virtualization capable:

Turning Hyper-V on via the following command:

bcdedit /set hypervisorlaunchtype auto

and the CPU will appear not to support virtualization again! Something, possibly a driver loaded by Hyper-V has masked certain information and preventing these tools from detecting that the CPU is virtualization capable.

The Model Specific Register (MSR)

Some research reveals that virtualization support may be enabled or disabled by writing to the Model Specific Register (MSR) located at address 0x3A. This MSR contains the following three bits:

Bit 0: lock bit
Bit 1: activate VMXON in SMX mode
Bit 2: activate VMXON outside of SMX mode

According to this website, the BIOS/EFI must set bits 1 and 2, or all three bits (including bit 0) so that VT-x support will be enabled. With Ubuntu 12.04 on my machine, installing the msr-tools package and executing the following command

rdmsr 0x3a

returns 5 indicating that virtualization is enabled. A value of 0 or 1 indicates that virtualization is disabled, in which case it can be enabled if it has not been locked (bit 0 is not set) by using the wrmsr command.

My guess is that a driver bundled with Hyper-V has written a value to this MSR to disable hardware virtualization. Since Hyper-V already knows that the machine supports virtualization, it proceeds to use the feature nevertheless. However, other tools simply read what the CPU reports and complains that virtualization is not supported. Some software such as VMware will automatically attempt to enable virtualization and makes use of it if the feature is not locked by the machine firmware and, for older versions of VMware, if the hv.enableIfUnlocked parameter in the .vmx file is set to TRUE.

Possible workarounds

Knowing the root cause of the problem, I proceeded to find tools to read/write MSR on Windows to change the value back to 5 and enable virtualization. Unfortunately such tools are hard to find since access to the MSR is limited to ring 0 (kernel-mode drivers) only on Windows. I found the following two applications - neither of which worked well:

Performance Inspector: a package of several performance inspector tools. One of which is MSR which supports reading/writing to model-specific registers. Unfortunately this tool refuses to install on Windows 8. Attempting to set compatibility mode to Windows 7 allows the tool to be installed, but the driver perfdd-win7.sys fails to be loaded with error 'The request is not supported'.It could however be due to 32-bit/64-bit compatibility issues.
RW-Everything: this tools appears to support reading/writing to the MSR. However, I could not locate the MSR register 0x3A to be edited among the list of MSR available:

Unless I could find another tool, the only possible way is to develop your own Windows tool to read/write the MSR. This would mean downloading the Windows Driver Kit and spend countless hours developing a kernel mode driver and an application to make use of the driver. The driver can then use the __readmsr and __writemsr methods to access the MSR values and setting the desired values. Obviously I am not going to attempt this - too much work to be done for too little value! Just disabling Hyper-V when there is a need for virtualization is sufficient for me.

Hardware virtualization on Mac systems

On a side note, the issue of hardware virtualization not being enabled by default seems to be a problem with earlier Mac Mini models. When this happens, you will need to use rEFIt to execute a custom EFI application that enables the feature. Before knowing that the problem was due to Hyper-V, I tried this on my Mac Mini and received an error "Image type IA32 is not supported by this X64 shell" because the EFI application was built for 32-bit Mac while my machine has 64-bit EFI, like most newer generation Macs.

On other Mac machines, hardware virtualization may be disabled by default, and can only be enabled by putting the machine to sleep for a few seconds and resuming it. On a multi-boot Mac system, if hardware virtualization appears to be enabled on Mac but not on Windows, try to start a Mac software that makes use of virtualization such as Parallels, and reboot the machine immediately to Windows without turning it off. According to this, the workaround will allow Windows to see that hardware virtualization is enabled and uses it. All these issues may be due to some bugs in the EFI booting process of the Mac that forgets to set the some required MSR bits.

Fortunately, my machine does not have these problems and hardware virtualization is always enabled natively, at least until Hyper-V is installed. I wonder why Hyper-V couldn't just leave the virtualization features enabled and allow other applications to run peacefully? This question is for the engineers at Microsoft to answer...

↧

Converting an Outlook Form Template (OFT) to a .NET Windows form in Visual Studio

July 18, 2013, 6:19 am

≫ Next: Exploring old Mac software on Mini vMac, Basilisk and SheepShaver

≪ Previous: Hyper-V and hardware assisted virtualization

In one of my work projects, I was assigned with the task of developing a standalone Windows form application to replace what was initially implemented as an Outlook Form Template (OFT) file using Visual Basic for Application (VBA) for data processing. The objective is to provide the user with an interface to fill in the required data and have them checked for validity before submitting.

Since the original form has over 500 fields, consisting of text boxes, dropdown lists, radio buttons and check boxes, it would be a nightmare to start designing the new form from scratch using Visual Studio form designer. My first attempt is to open the original form in Design mode in Outlook, copy the fields and paste to Visual Studio form designer. This did not work - probably because the clipboard format is different. I decided to find a method for me to copy the fields from the original Outlook form over to save time.

Importing the form in Visual Studio

I came across this MSDN article indicating the possibility of importing an Outlook form to be used in Visual Studio, and decided to attempt it.

First open the form in Design mode in Outlook:

Add a new Outlook form region and pasted all controls from the original form to the region:

After that, save the Outlook form region to an .OFS (Outlook Form Storage) file on the hard drive.

Finally, in a Visual Studio Outlook add-in project, add a new Outlook Form Region and choose to import from an existing Outlook Form Storage file:

Most of the wizard settings can be kept as default, except for the "Select the type of form region you want to create page"and the "Which custom message classes will display this form region"option, which should be Replace-all and IPM.Task.XXXX where XXXX is any valid name for your form region respectively.

After completing the wizard, a set of form region files, including the .designer.cs file, were added to the project.

Designer support for imported form regions

However, to my disappointment, despite the presence of the designer class, Visual Studio did not open this form region in the designer and simply opened the code editor for the designer class file. If however in the wizard I chose to design a new form region, Visual Studio would allow me to design the form region user interface. The difference is shown in the icons of the form region in the Solution Explorer:

Although this seems to be a common problem and can usually be fixed by reopening Visual Studio, cleaning and rebuilding the solution, in this case, I could not get FormRegion1 to open in the designer despite trying various workarounds.

The difference seems to be in the base class of the 2 forms. FormRegion2 inherits from Microsoft.Office.Tools.Outlook.FormRegionBase whereas FormRegion1 inherits from Microsoft.Office.Tools.Outlook.ImportedFormRegionBase and cannot be opened in the Designer. In fact while the generated for FormRegion2 contains all the necessary information to render the form fields at runtime, the code for FormRegion1 only contains minimal type declaration for the form controls, with most other information being retrieved from the .OFS file at runtime. This explains why the designer does not open FormRegion1 - it simply cannot be manipulated easily this way.

partial class FormRegion1 : Microsoft.Office.Tools.Outlook.ImportedFormRegionBase
{
        private Microsoft.Office.Interop.Outlook._DRecipientControl to;
....
        protected override void InitializeControls()
        {
            this.to = (Microsoft.Office.Interop.Outlook._DRecipientControl)GetFormRegionControl("To");
....
        }
....
        [System.Diagnostics.DebuggerNonUserCodeAttribute()]
         byte[] Microsoft.Office.Tools.Outlook.IFormRegionFactory.GetFormRegionStorage(object outlookItem, Microsoft.Office.Interop.Outlook.OlFormRegionMode formRegionMode, Microsoft.Office.Interop.Outlook.OlFormRegionSize formRegionSize)
        {
            System.Resources.ResourceManager resources = new System.Resources.ResourceManager(typeof(FormRegion1));
            return (byte[])resources.GetObject("Survey");
        }
....
}

At this point it became clear to me that it would not be possible to open the imported FormRegion1 in the Designer and I had to come up with a different method to copy the form fields.

Exporting the form controls

I then had an idea of enumerating through the form controls at run-time and outputting them to a format readable by Visual Studio. The best candidate for the file format would be VB6 form (.frm) file. This format is human-readable, can be easily constructed from code and can be upgraded to a .NET Windows form using the VB6 upgrade wizard in Visual Studio 2008 and earlier.

The best place to do this would be in the FormRegionShowing event:

private void FormRegion1_FormRegionShowing(object sender, System.EventArgs e)
{
UserForm oForm = this.OutlookFormRegion.Form;
foreach (Control ctl in oForm.Controls)
{
            if (ctl is Outlook.OlkCommandButton)
            {
                Outlook.OlkCommandButton cmdButton = (Outlook.OlkCommandButton)ctl;

                string buttonTemplate = String.Format(@"
    Begin VB.CommandButton {0}
        Caption         =   ""{1}""
        Height          =   {2}
        Left            =   {3}
        TabIndex        =   {4}
        Top             =   {5}
        Width           =   {6}
    End",
                    ctl.Name.Replace("", ""), escapeString(cmdButton.Caption), (int)(ctl.Height * 20), (int)(ctl.Left * 20), count, (int)(ctl.Top * 20),
                    (int)(ctl.Width * 20));
.............
            }
}
}

The generated .frm code for the button would look like:

   Begin VB.CommandButton Command1
      Caption         =   "Command1"
      Height          =   855
      Left            =   480
      TabIndex        =   3
      Top             =   3480
      Width           =   2175
   End

Using this method I was eventually able to generate the VB6 equivalent of all the forms in the original Outlook template and ugprade them to .NET forms. With some modifications, the upgrade forms are ready for further development work. The VBA code-behind of the forms can be upgraded using Telerik only code converter tool.

Potential issues with the conversion

Although it works well enough for my needs, this method is far from perfect. Differences in the possible measurement units (pixel, points, twips) used by Outlook and VB6 for the form fields cause the generated VB6 forms to look slightly different from the original form. Also, some imported controls belong to the namespace Microsoft.Vbe.Interop.Forms while others belong to the Microsoft.Office.Interop.Outlook namespace. This results in the need to use dynamic data type in my code to avoid unnecessary type casting. Finally, items in a combo box are not included in the .frm file, but stored in a separate .frx file:

   Begin VB.ComboBox Combo1
      Height          =   315
      ItemData        =   "Form1.frx":0000
      Left            =   3960
      List            =   "Form1.frx":0010
      TabIndex        =   5
      Text            =   "Combo1"
      Top             =   480
      Width           =   1215
   End

Since the frx file is binary, I did not attempt to generate it and simply copy the combo box items manually. For simplicity, several other less common form control properties are also not migrated and are set manually after the conversion process.

The prototype code to generate the VB6 form can be downloaded here for those who are interested.

↧

Exploring old Mac software on Mini vMac, Basilisk and SheepShaver

August 23, 2013, 6:16 am

≫ Next: Interfacing TEA5767 FM Receiver with PIC using I2C

≪ Previous: Converting an Outlook Form Template (OFT) to a .NET Windows form in Visual Studio

It has been a few years since I last experimented with old Macintosh systems on my PC using Mini vMac, Basilisk, and SheepShaver due to work and other commitments. When I decided to start these virtual machines again recently, I am still amazed to see the user experience these operating systems provide and their capabilities given the limited hardware configuration at the time. This article will showcase some old Mac applications that left an impression on me as well as some of my interesting findings.

Original Mini vMac

The original release of Mini vMac (downloadable from http://minivmac.sourceforge.net) emulates a Macintosh Plus that has 4MB of RAM running on a 8MHz Motorola 68000 processor. It can run up to System 7.5.5, available as free download from Apple at http://www.info.apple.com/support/oldersoftwarelist.html

A wide range of office software was available for System 7.5.5:

WordPerfect Works version 1.2, MacDraw, MacPaint and The Print Shop, a banner/letter/card printing utility:

What is the difference between a Draw and a Paint application, e.g. between MacDraw and MacPaint? Many computer users nowadays would not be able to answer this question. The basic difference is that, a Draw application stores shapes as vectors so that they won't be pixelated and can be further manipulated once the drawing is saved. A Paint application, however will store shapes as pixels and any further manipulations will have to be done pixel by pixel.

PowerPoint and Sum-It, a spreadsheet utility:

An HTML editor for the Plus, in black and white. Try to use HTML5 and CSS on this.... :)

MacWeb is among the few web browsers that can run on the Plus. As Mini vMac does not emulate a network card, it can only open local HTML files:

A nice looking clock on Mini vMac:

Despite the low configuration, various multimedia applications were available. Imagine designing posters and flyers using Photoshop on a black and white screen:

It can open modern graphics files also. The following is a photo of the Eiffel Tower, opened under GIF Converter:

This is probably very hard to find nowadays - MacroMind MusicWorks allows you to compose songs:

And MacroMind VideoWorks allows you to design nice animations that are playable on the Plus:

Another VideoWorks video, this time with audio:

Surprisingly there is a text-to-speech tool called Voice Box available for the Plus. After much effort I managed to grab a working version and make the Plus speak. See the following video and notice how the application says "Goodbye" when you choose to quit.

It is amazing considering that the audio on a Macintosh Plus is generated using a 8-bit DAC at 22kHz sampling rate. The PC did not come bundled with sound hardware, other than the PC speaker, until years later.

The above clips were captured using CamStudio for Windows. Although CamStudio supports recording of audio output from the speaker, attempting to select Option > Record audio from speakers results in an error message "WaveoutGetSelectControl() failed.". This error is mentioned in the official website, with no solution available. I also tried a suggestion of recording from the microphone using the Stereo Mix option, only to find that no such option exists on my machine. It could be due to Windows 8, or due to the Mac Mini Bootcamp drivers for Windows. In the end, I used a stereo audio loopback cable that connects the speaker output port to the line-in port, and the audio was recorded perfectly.

What would a computer be without some games? Well, there were plenty of games, both text and graphics, for the Macintosh Plus. I downloaded many of them from an abadonware website and install on my vMac:

Bouncing Ball, Cross Word, the famous Hangman, and StuntCopter in particular:

And who could forget the famous Prince of Persia game. It comes with sound too, on the Macintosh Plus:

The screenshots show the game progress from the start of Level 1. The graphics still look nice and real on a monochrome monitor! Of particular note is the copyright protection where the prince has to drink a potion matching the first letter of a specific word in the manual before he can proceed to the next level. In those days a paper manual was all you need to prove ownership of a game, no product key, no server validation and what not! Unfortunately time has changed and I was able to provide a cheat sheet online describing the expected letter:

There was a long jump at the beginning of level 2. On a PC, you would need to stand right at the ledge and press Ctrl, Shift and the arrow key. The prince would then jump to the other side, hold on to the ledge and climb up. However, on mini vMac, the Ctrl key is the host key which activates Mini vMac control mode. To jump across the gap, you need to press Ctrl-K to enable Emulated Control Key mode, and then use either the Alt key or the Windows key to jump longer.

You will need to fight against several guards after this screen. Take that that since the emulated SHIFT key seems to get stuck after a while when running Mini vMac on Windows 8, causing the prince to walk slowly or unable to fight, it is better to use a Windows XP or a Windows 7 machine to enjoy a smooth and playable game.

Somewhere near the end of level 2 you will notice 2 potions, one is poisonous while the other increases your strength. Which one to drink? On a color monitor, they would be indicated by different color. However, on the Macintosh Plus monochrome monitor, you simply have to take a guess:

The corresponding screen when playing on Dosbox. The red potion on the right is the one the prince should drink!

Talking about Dosbox, is there a similar possiblity of running PC apps on the Macintosh Plus? Yes! This is made possible by Soft PC which emulates a 80286 with 640KB of RAM running MS DOS 3.30, as reported by Norton Commander's System Information and Microsoft MSD utility:

The left screenshot shows SoftPC starting up on System 7.5.5, which is considered as an abnormal situation reported by Mini vMac. What exactly is abnormal? Is it because we are simply running MS-DOS on a vintage Mac? I guess the answer is due to Soft-PC trying to initialize or access some hardware not emulated by Mini vMac.

Despite the limited configuration, a variety of games can run and are playable on SoftPC, such as AlleyCat and ParaTrooper:

Dyna Blaster cannot run and simply causes a screen distortion, resulting in me having to restart Mini vMac:

Imagine how you would design Turbo Pascal units and work with Microsoft Visual Basic for DOS's form-based applications under a 512×342 monochrome screen:

How did I get these DOS applications to appear in Soft PC in the first place? Soft PC provides three ways, configurable from the Setup menu, for the emulated PC to access data, via floppy disk, hard disk, or a network drive. Emulating a floppy drive requires access to a physical floppy drive, which is not emulated by Mini vMac. When setting up a new emulated hard drive, Soft PC will create a hard drive image file on the system drive. I tried to use HFVExplorer copy the disk image to my PC, use WinImage to add files to the image and import it back to the emulated Mac. However, it does not work because the Mac resource fork is lost during the copying process, and the modified image is not recognized by SoftPC as a hard disk image.

The easiest way is to use HFVExplorer to copy DOS applications onto a folder in the Mac hard drive and configure Soft PC to emulate a network drive pointing to that folder. As long as the folder does not have a lot of files and approach DOS 3.3 file system limitation, it will work.

Soft PC also emulates a PC mouse. However, when mouse emulation is enabled, the speed is too slow to be used. The DOS keyboard mapping in Soft PC is also weird and I am not sure where to configure it correctly, other than by trial and errors. Some DOS applications also appear to run in monochrome, but perhaps due to poor design, it is difficult to distinguish between certain screen elements (e.g. selected menu with a different background color).

MacShell, a terminal for System 7.5.5, albeit not useful because it only accepts simple command like man:

Finally,an attempt to run Mini vMac under Mini vMac, which fails due to lack of memory:

Custom Mini vMac with more RAM and colors

The following shows System 7.5.5 running on Mini vMac with 256 colors. This version of Mini vMac is taken from a set of custom-built versions of Mini vMac that have up to 8MB of RAM and 16-bit colored graphics (download a copy here)

Turbo Pascal under Soft PC now presents itself with it classical white-on-blue IDE:

Prince of Persia works well and looks similar to its PC counterpart under a 8MB Mini vMac with 256 colors:

With the added RAM, Prince of Persia II can now run beautifully with colors:

Photoshop now works in colors too:

Unfortunately, with 8MB of RAM, Soft PC refuses to start up, complaining Divide Overflow error. Does this sound like Turbo Pascal's Runtime Error 200 where the CPU speed exceeds 200MHz?

Mini vMac appears to run this time, complaining about missing ROM file:

However, even with the correct ROM image and hard disk, the emulated Mac could not boot up and just show a black screen. My guess is that the failure could be due to the lack of capabilities in the emulated video card.

BasiliskII

Next on our list is Basilisk II which can run up to System 8.0.1 on a 68k Mac:

It can also run System 7.6.1 in color:

Microsoft Excel, PowerPoint and Word 6.1, the last version that can run on System 8:

Basilisk emulates a network card and makes it possible to access the Internet from System 8. To do this, you will need to install the custom network driver provided by Basilisk as a protocol for the network card which you are using to access the Internet. The driver is supported under Windows 7 and older only. On Windows 8, Microsoft forces users to use signed drivers and does not allow the installation of unsigned drivers as a network protocol even under test mode and will just show an error:

iCab, IE4 and WannaBe browsers for System 8, if you managed to get Ethernet to work:

Most of these old browsers are too slow and buggy to be used on modern websites, due to the introduction of new HTML elements that are not understood by the browsers. In the above example, IE4 displayed the closest representation of Wikipedia that would be shown by a modern browser.

Unfortunately, I did not have the chance to try the Audio CD player in System 8 this time. It used to work with an audio CD when I tried it years back, but my Mac Mini now does not have a CD/DVD drive and Basilisk's CD-ROM driver does not work with a virtual CD-ROM driver emulated by Daemon Tools or other similiar tools.

SheepShaver

System9.0.4 is the last classic Macintosh version that can officially be emulated on a PC using SheepShaver. This is because System 9.1 requires the Memory Management Unit (MMU), which is not emulated by SheepShaver. Recently I found a post which claims that it is possible with some modifications (and perhaps some associated problems). I have not tried this, however.

The About screen of System 9.0.4 under SheepShaver:

Microsoft Word 98, the last version to support System 9, with the well-known Office Assistant, running on SheepShaver:

Classilla is the only actively maintained web browser that still supports System 9 till this day. It sends a mobile user agent by default so that most web servers will return less heavy content suitable for low-end devices. In my experience, it is quite fast and displays many modern web pages nicely, including Wikipedia:

The equivalent web browser on a PC would be Dillo or D+ Web Browser. I use them on my Windows 98 machine with just 32MB of RAM to browse the modern web and they work quite smoothly.

This is a screenshot of MacWeb browser when running on System 9 showing Google search page. How much junk have we added to the Internet? HTML5, CSS, JavaScript, Flash and whatnot. All that needs to be shown for a Google home page is just a text box and two buttons:

I also tried the Mail application (which is actually Outlook Express) that comes with System 9. Unfortunately, it does not work with modern mail services such as Hotmail, Yahoo Mail or GMail. Although the software seems to support SSL and SMTP authentication, attempt to configure and log on to the mail server would simply crash SheepShaver, hang at "Connecting" forever, or report authentication errors. I guess the problem is related to SSL authentication, but without the time to verify and confirm this, the only way to get the Mail application to work is to find a provider that doesn't require SSL (which is very hard nowadays) or to use a local mail server such as hmail and configure the server to accept unencrypted connections.

The last screenshot in this article puzzles me. I use MacWeb from System 9 to browse Wikipedia and always receive an error page indicating "Unconfigured Domain". The error is persistent and Wikipedia website is still accessible from other browsers, including other System 9 browsers. Why is this the case since browsers only request for HTML content from the server and render them? While older browsers may fail to display modern contents, I would never expect them to display a domain error message for a totally functional site. This question is left as an exercise for the readers.

↧