Download

secondscreen_setup.exe

Despite the effort to make the best of the watch, it's still just a good hack. So, some instabilities linked to data overflow and mishandled packets can create random deconnexions and freezes of the LiveView.

Source included


Found buried in a thrift store, here is an artefact from the pre-smartwatch era, the famous Sony Ericsson LiveView (2011). No internal memory, no Android but designed to work with, no LTE-Bluetooth. Not real touchscreen either. An in my case, the battery being dead, my range of possibilities is reduced to a permanent sector plug (USB). Nothing big enough to discourage me, in the end.

Searching for clues about LiveView dev brings me to this documentation (pdf) along with LiveViewLib (C#), successful amalgam of multiple projects like LiveViewNet. No real reverse-engineering to do today.

Second Screen : a first project

The screen in itself is a nice 128 square pixels. Let's use all that huge surface to display pictures. Here is the logic:

  • Connect to the stuff
  • Force it to stay turned on (the screen fade off after some seconds)
  • Send the image
  • Profit
  • (Start again?)

Connection

Pretty simple, done thanks to the indispensable 32Feet Bluetooth library from InTheHand.

blah, blah...

server = new LiveViewServer();
server.connectListener += new LiveViewServer.ConnectListener(server_connectListener);
try{
    server.Start();
...

Once connected, we use a tiny handler to send the first commands:

server.connectListener += (l) =>{
    l.Send(new GetTimeResponse(GetTimeResponse.GetLocalTime(), true)); //plus stable quand synchronisé avec l'heure dès le départ
    l.Send(new SetMenuSizeMessage(0)); //on ne peut envoyer du bitmap que quand il n'y a aucun item de menu
    l.SendBytes(wakeUp); //pour le maintenir allumé

I had to add a SendBytes function to the library to send raw data because the official screenOn one, provided by the official SDK, seemed to not have been implemented. Indeed, by sending

l.Send(new SetScreenmodeMessage(0, Screenmode.BRIGHTNESS_OFF));
l.Send(new SetScreenmodeMessage(0, Screenmode.BRIGHTNESS_DIM));
l.Send(new SetScreenmodeMessage(0, Screenmode.BRIGHTNESS_MAX));

meaning

// LiveViewLib.MessageTypes.SetScreenmodeMessage
// 40-04-00-00-00-01-61
// LiveViewLib.MessageTypes.SetScreenmodeMessage
// 40-04-00-00-00-01-63
// LiveViewLib.MessageTypes.SetScreenmodeMessage
// 40-04-00-00-00-01-65

the screen kept fading.

I then decided to send the packets as Byte[], making the brightness byte vary (0x61, 0x63 and 0x65 here). As I was playing with the values, I sent the 0x64 one and discovered with joy that it worked! Bug or legit byte, we now have a wake-up + always on packet.

readonly byte[] wakeUp = { 0x40, 0x04, 0x00, 0x00, 0x00, 0x01, 0x64 };

Sending a picture

As specified by the doc, sending tiny pics like 48x48, 36² or 16² is easy, however I never succeded to send a 128² one by the regular way. I then created a function which:

  • Takes the input picture
  • Center (fit & crop) it on 128x128 max
  • Split it in 32x32 tiles
  • Display the blocs to matching coordinates
Image img = Image.FromFile("lenna.jpg");
//resize, crop and fit to lv size thus sq^128
int w = img.Width < img.Height ? 128 : img.Width * 128 / img.Height;
int h = img.Height < img.Width ? 128 : img.Height * 128 / img.Width;
//center pic
int dW = w > 128 ? (w - 128) / 2 : 0;
int dH = h > 128 ? (h - 128) / 2 : 0;
img = img.GetThumbnailImage(w, h, null, IntPtr.Zero);

//random tile order
var mosaic = Enumerable.Range(0, 16).OrderBy(n => Guid.NewGuid());
foreach (int tile in mosaic) //split in 32*32 tiles to avoid 128*128 buffer f*ckery
{
    int y = tile / 4; //clockwise
    int x = tile % 4;
    Image o = new Bitmap(32, 32);
    using (Graphics g = Graphics.FromImage(o))
    {
        g.DrawImage(img, new Rectangle(0, 0, 32, 32), new Rectangle(x * 32 + dW, y * 32 + dH, 32, 32), GraphicsUnit.Pixel);
    }
    l.Send(new DisplayBitmapMessage((byte)(x * 32), (byte)(y * 32), imageToByteArray(o))); //sending the puzzle piece
}
img.Dispose();

Thus, the image appears the old goode "My Pictures"' Windows XP screensaver.

Now have the abilty to display a picture correctly we're just lines away to transform the LiveView into a digital photo frame.

//getting all my jpg pictures as shuffled
var pics = Directory.EnumerateFiles(Environment.GetFolderPath(Environment.SpecialFolder.MyPictures), "*.jpg", SearchOption.AllDirectories).OrderBy(n => Guid.NewGuid()).ToList();

Text = " Photo Frame Mode ("+pics.Count+" pictures)"; //connection ok
for (int i = 0; i < pics.Count; i++)
{
    Image img = Image.FromFile(pics[i]);
    //resize, crop and fit to lv size thus sq^128
    int w = img.Width < img.Height ? 128 : img.Width * 128 / img.Height;
    int h = img.Height < img.Width ? 128 : img.Height * 128 / img.Width;
    //center pic
    int dW = w > 128 ? (w - 128) / 2 : 0;
    int dH = h > 128 ? (h - 128) / 2 : 0;
    img = img.GetThumbnailImage(w, h, null, IntPtr.Zero);

    //random tile order
    var mosaic = Enumerable.Range(0, 16).OrderBy(n => Guid.NewGuid());
    foreach (int tile in mosaic) //split in 32*32 tiles to avoid 128*128 buffer f*ckery
    {
        int y = tile / 4; //clockwise
        int x = tile % 4;
        Image o = new Bitmap(32, 32);
        using (Graphics g = Graphics.FromImage(o))
        {
            g.DrawImage(img, new Rectangle(0, 0, 32, 32), new Rectangle(x * 32 + dW, y * 32 + dH, 32, 32), GraphicsUnit.Pixel);
        }
        l.Send(new DisplayBitmapMessage((byte)(x * 32), (byte)(y * 32), imageToByteArray(o))); //sending the puzzle piece
    }
    img.Dispose();

    Thread.Sleep(15 * 1000);
}
Environment.Exit(0);

Using it as a second screen display

Thanks to the power to display any picture, let's give it some VNC aspiration by giving it the ability to display a humble part of our screen.

We create a Form of which the internal zone (Client) will be 128², place into its top left corner an invisible picturebox as absolute screen reference point. The client background is transparent thanks to the magenta trick.

        public Image grabScreenPart()
        {
            Image img = new Bitmap(128,128);
            using(Graphics g = Graphics.FromImage(img)){
                g.CopyFromScreen(captureBox.X, captureBox.Y, 0, 0, new Size(128, 128));
            }
            return img;
        }

and, in a loop, we make a snapshot of the Form Client area, before send it to the LiveView. That every 15 seconds (pretty nice framerate, isn't it?).

            for(;;)
            {
                if (isMoving) continue; //don't refresh when form moved to avoid overflow

                Image curPic = grabScreenPart();
                var mosaic = Enumerable.Range(0, 16); //.OrderBy(n => Guid.NewGuid());
                foreach (int tile in mosaic) //split in 32*32 tiles to avoid 128*128 buffer f*ckery
                {
                    int y = tile / 4; //clockwise
                    int x = tile % 4;
                    Image o = new Bitmap(32, 32);
                    using (Graphics g = Graphics.FromImage(o))
                    {
                        g.DrawImage(curPic, new Rectangle(0, 0, 32, 32), new Rectangle(x * 32, y * 32, 32, 32), GraphicsUnit.Pixel);
                    }
                    liveView.Send(new DisplayBitmapMessage((byte)(x * 32), (byte)(y * 32), imageToByteArray(o))); //sending the puzzle piece
                }

                System.Threading.Thread.Sleep(15*1000); //resting a bit
            }

All the screen part delimited by the window will be displayed on the watch. Useful to follow some vital informations while we're in the next room!

Going further: MusicPeek Media Player integration

Once the LiveView connected to the computer while MusicPeek is running, it's gonna display the current cover art and allow the user to switch tracks by swiping the LiveView's screen. For some obscure reasons we got a memory leak once the device paired, but I hope to fix that problem in a future build.

Previous Post Next Post