How to handle long SMS text from Nexmo API properly?

Question

be kind, this is my first question and my english is not very good.

I have no problem to deal with standard messages from Nexmo API and I want to receive long SMS just like I do with standard ones (i.e. in one block).

Exemple of data received from Nexmo for a standard SMS:

    $_GET['msisdn'] ==> "33612345678" // "from"
    $_GET['to'] ==> "33687654321"
    $_GET['messageId'] ==> "02000000478EBE09"
    $_GET['text'] ==> "Hello world!"
    $_GET['type'] ==> "unicode"
    $_GET['keyword'] ==> "HELLO"
    $_GET['message-timestamp'] ==> "2014-11-25 14:06:58"

Long one: (Nexmo sent it piece by piece)

    $_GET['msisdn'] ==> "33612345678" // "from"
    $_GET['to'] ==> "33687654321"
    $_GET['messageId'] ==> "02000000478EBE09"
    $_GET['text'] ==> "the first part of a too long text..."
    $_GET['type'] ==> "unicode"
    $_GET['keyword'] ==> "THE"
    $_GET['message-timestamp'] ==> "2014-11-25 12:06:58"
    $_GET['concat'] ==> "true"
    $_GET['concat-ref'] ==> "108" // unique identifier for long SMS text
    $_GET['concat-total'] ==> "4" // or more, or less...
    $_GET['concat-part'] ==> "1" // index of the part, start at 1

See more on the Nexmo documetation: Here

So, I started from a library found on github (Nexmo-PHP-lib) and did this: (pretty ugly but it's for test purpose)

public function inboundText( $data=null ){
    if(!$data) $data = $_GET;

    if(!isset($data['text'], $data['msisdn'], $data['to'])) 
        return false;
    if(isset($data['concat']) && $data['concat'])
    {
        session_start();
        if ($data['concat-part'] > 1) // first part ?
        {
            if ($data['concat-total'] == $data['concat-part']) // last part ?
            {
                // last part ! stock the data in the text and unset now useless $_SESSION entry!
                $data['text'] = $_SESSION[(string)$data['concat-ref']] . $data['text'];
                unset($_SESSION[(string)$data['concat-ref']]);
            }
            else // not the first or the last, so concat !
            {
                // concat the new part in the entry named after the concat-ref
                $_SESSION[(string)$data['concat-ref']] .= $data['text'];
                return false;
            }
        }
        else // first part ! so creat a $_SESSION entry for that! (named after concat-ref)
        {
            $_SESSION[(string)$data['concat-ref']] = $data['text'];
            return false;
        }
    }
    // Get the relevant data
    $this->to = $data['to'];
    $this->from = $data['msisdn'];
    $this->text = $data['text'];
    $this->network = (isset($data['network-code'])) ? $data['network-code'] : '';
    $this->message_id = $data['messageId'];

    // Flag that we have an inbound message
    $this->inbound_message = true;

    return true;
}

It works perfectly with local test but not when it's hosted on my heroku server, the $_SESSION array seems to be reset at each part of the sms...

So, have you any idea of how to handle it properly? (and without an ugly temporary SQL table). How can I retreive the previous part of the message until I receive it completely?

Answer 1

Using a session to store the temporary sms parts would only work if there is an exchange of keys between the client and server with each HTTP request which identifies the session.

In PHP, when you create a session and store a value inside $_SESSION a file is created on the server to store this data (unless you are using a DB session handler, in which case the session data is stored in a database). This data is referenced with an identifier (e.g., PHPSESSID=66dda288eb1947843c2341b4e470fa28) which is normally provided to the client as a cookie. When the client returns at the next HTTP request, the identifier is returned to the server as a cookie value, and the server uses this to reference the same session data.

While your server may be providing the cookie to the Nexmo client when it connects to your endpoint URL, Nexmo is probably not storing the cookie and returning it with its next request. (This is an assumption on my part, but I think it is a safe one—although I can't explain why it works on your local machine but not Heroku. Anyways, it is easy to test—just check if the Nexmo client provides any $_COOKIE value during subsequent requests).

Bottom line: if the Nexmo client doesn't use cookies to save state between requests, you cannot use sessions to save temporary sms parts.

A better option, in any case (because the temporary saved message parts would persist in case of a server reboot), would be to save each temporary part to a small database table. (Don't worry, it's a beautiful temporary SQL table. ;) Here's an example using MySQL:

CREATE TABLE `response_concat_tbl` (
    `concat-ref` SMALLINT NOT NULL DEFAULT '0',
    `concat-total` TINYINT UNSIGNED NOT NULL DEFAULT '0',
    `concat-part` TINYINT UNSIGNED NOT NULL DEFAULT '0',
    `text` VARCHAR(160) NOT NULL DEFAULT '',
    `added_datetime` DATETIME DEFAULT NULL,
    UNIQUE KEY `concat-ref` (`concat-ref`, `concat-part`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='A place to temporarily store multipart SMS messages before they are combined and saved as a whole response.';

Here I've defined a UNIQUE KEY so we can avoid duplicate message parts with a REPLACE INTO clause (in case the Nexmo client tries to submit the same part twice). The added_datetime can be used to clean up orphaned messages in the future (in case the final part of a message is never received).

Now let's insert some sample data:

REPLACE INTO response_concat_tbl (`concat-ref`,`concat-total`,`concat-part`,text,added_datetime) VALUES ('101','4','1','This is',NOW());
REPLACE INTO response_concat_tbl (`concat-ref`,`concat-total`,`concat-part`,text,added_datetime) VALUES ('101','4','2','a multipart',NOW());
REPLACE INTO response_concat_tbl (`concat-ref`,`concat-total`,`concat-part`,text,added_datetime) VALUES ('101','4','4','for you!',NOW());
REPLACE INTO response_concat_tbl (`concat-ref`,`concat-total`,`concat-part`,text,added_datetime) VALUES ('101','4','3','sms message',NOW());

Now we can use MySQL's GROUP_CONCAT function to grab all the parts in one query.

SET SESSION group_concat_max_len = 1000000;
SELECT GROUP_CONCAT(text ORDER BY `concat-part` ASC SEPARATOR ' ') AS text
FROM response_concat_tbl
WHERE `concat-ref` = '101'
GROUP BY `concat-ref`;

We set the group_concat_max_len setting so the total string length may be longer than the default 1024 characters (although that's already lots of message). Here's the result:

+-------------------------------------------------------------+
| GROUP_CONCAT(text ORDER BY `concat-part` ASC SEPARATOR ' ') |
+-------------------------------------------------------------+
| This is a multipart sms message for you!                    |
+-------------------------------------------------------------+

If you're not using MySQL, you might need to do a bit more work (some duplication check and then a loop) without REPLACE INTO and GROUP_CONCAT.

Here is a full working example using this technique:

class SMS
{
    static public function processMultipart($sms)
    {
        $db =& DB::getInstance();

        if (isset($sms['concat']) && $sms['concat']) {
            // This sms is part of a multipart message, save it temporarily to the db.
            $sms = array_map('trim', $sms);
            $db->query("
                REPLACE INTO response_concat_tbl (
                    `concat-ref`,
                    `concat-total`,
                    `concat-part`,
                    `text`,
                    `added_datetime`
                ) VALUES (
                    '" . $db->escapeString($sms['concat-ref']) . "',
                    '" . $db->escapeString($sms['concat-total']) . "',
                    '" . $db->escapeString($sms['concat-part']) . "',
                    '" . $db->escapeString($sms['text']) . "',
                    NOW()
                )
            ");

            if ($sms['concat-total'] > $sms['concat-part']) {
                // Not all the parts have been received; return false to signal the fact we don't have a complete message yet.
                return false;
            }
            // Otherwise, it means the last part has just been received. Concatonate all the parts and return it.

            // Increase the max length returned by MySQL's GROUP_CONCAT function.
            $db->query("SET SESSION group_concat_max_len = 32000");

            // Group the sms responses by concat-ref and return them as a concatonated string.
            $qid = $db->query("
                SELECT GROUP_CONCAT(text ORDER BY `concat-part` ASC SEPARATOR ' ')
                FROM response_concat_tbl
                WHERE `concat-ref` = '" . $db->escapeString($sms['concat-ref']) . "'
                GROUP BY `concat-ref`
            ");
            list($sms['text']) = $db->fetch_row($qid);

            // Delete the temporary records.
            $db->query("
                DELETE FROM response_concat_tbl
                WHERE `concat-ref` = '" . $db->escapeString($sms['concat-ref']) . "'
            ");
        }

        // If this is not a multipart message, the original sms data is returned. If it is a multipart message, we're returning the fully-concatonated message.
        return $sms;
    }
}

And here's how to use this function:

// False is returned here if we need to wait for additional parts to arrive. Otherwise, $sms is populated with the final, usable data.
if (false === $sms = SMS::processMultipart($_GET)) {
    header($_SERVER['SERVER_PROTOCOL'] . ' 200 OK', true, 200);
    die('Waiting for additional message parts');
}

// Do something with $sms['text'], e.g.,
SMSResponse::save($sms['text']);