Web/Tech

iPhone iWant

iPhone iWantFinally installed Typepad on my iPhone. That makes one less excuse to not post over here on Downstream more often. But what would really turn me into a mobile blogging maniac is an app that can translate this photo into a UPC. Next time I blow out candles that's what I'm wishing for.

October 04, 2008 in Web/Tech | | Comments (0)

| | |

CEC Convenes in Las Vegas

Sun's annual Customer Engineering Conference kicks off today with two themes in which our prized field organization will be immersed for the next three days: Green Computing and "Redshift" - the Internet growth trajectory that is underserved by Moore's law.

I had the privilege to interview Matt Ingenthron last Friday in a prelude to the event, in which we ad lib'ed a short repartee about the relevance of Sun to developers who are doing PHP and Ruby applications, and who are classically building on LAMP.  These developers are a decidely Redshifted bunch and are eminently entitled to the bounty of advantages for Redshifted apps of Solaris, and Sun's SPARC and X64 systems, so we give it to them...

There will plenty of news about Sun's successes in the Redshift ecosystem pouring out of the Nevada desert over the next three days, so stay tuned.

October 08, 2007 in Web/Tech |

| | |

Siguente Drupalcon - Día Dos

Sometimes the relevance of a technology only becomes apparent after examining it's origins.  For the uninitiated, I'll drop back a few sentences here to help understand Drupal's roots, before talking about some of the interesting things people are doing with it. 

Drupal logo

Drupal was created by Dries Buytaert in 2001.  Drupal is licensed under the GPL and is written in  PHP.  It is typically deployed in a LAMP environment although a few prominent sites have deployed Drupal on Solaris.  Drupal 5 is the current release with Drupal 6 due out in fall of 2007.

The popularity of Drupal has been driven by a passionate group of developers who come from very diverse backgrounds and who have applied Drupal to an equally diverse range of needs. 

The early community of Drupal developers grew primarily through rapid adoption by Web 2.0 hobbyists, nonprofits, and political activist organizations.  Howard Dean's push for the U.S. Presidency in 2004, noted for leveraging online tools and communities, owes much of its success to the Drupal powered 'Deanspace'.  This highly visible use of Drupal drew much attention to the community and triggered a huge wave of Drupal adoption. 

One of the most interesting uses of Drupal that I've seen is that which was presented at DrupalCon by Ivan Labra from SPAWAR, the Space and Naval Warfare division of the U.S. DoD.  He is using it as a integration platform in support of SPAWAR's peace and stabilization efforts, in which basic integrated ICT capability must be deployed into austere and sometime unstable environments.  Known as Speed-to-Capability, this project defines a technical architecture and deployment strategy for quickly building communication and collaboration capacity FOSS components combining PBX (Asterisk), Instant Messaging/Chat (Jabber), email (Postfix) and software provisioning (HostMaster) capabilities on the Drupal framework.   In my role advising on technology capacity in the developing world, I hope to work with Ivan in the future to apply this important communication and collaboration capacity.


September 20, 2007 in Web/Tech |

| | |

Siguente Drupalcon - Día Uno

Day one of DrupalCon Barcelona 2007 is over, but my jet lag is not.  I did manage to stay awake for the entire day, but only had time to attend two sessions:

  • OpenID: It's in core... now what?   by James Ransom Walker.   James is clearly an OpenID advocate and says the risks associated with it are manageable, or at least acceptable.  OpenID has been added to the Drupal 4.7, with updates for 5 and 6 coming soon (I'm not sure whether that's and Iraq pullout-style timetable, or a clever call for volunteers to lend a hand - James did say he could use some help).  This much heralded addition to Drupal gives developers an "out of the box" provider and  relying party status if they want it. It also comes with a new set of concerns for developers whose permutations are myriad: What is the trust model I want to deploy?  What level of protection do my users need from my provider service? As a relying party, what level of authentication do I need from a provider?  How do I choose to providers to accept?   Do I care whether a user's ID is globally unique *forever*, or just for now?  The OpenID spec. itself leaves the developer with all of these choices and more.  OpenID's flexibility is both a virtue and a failing.  Maybe someone in the DrupalCon Barcelona 2007 logoOpenSSO community can lend a hand to James and avert the sedimentation of a partial solution to an omnipresent problem.  OpenSSO is moving quickly to support OpenID provider implementations.  It has support for the relevant federation standards, and it even has a PHP Client SDK and a PHP library for SAML 2.0 Relying Party.  When it comes to Identity Management, I'm not convinced that today's "good enough" won't be tomorrow's compliance regulation headache or M&A due diligence hiccup.  My vote is for an OpenSSO based identity module for Drupal 5 and 6 rather than an OpenID only module.
  • "Enterprise" Drupal  by Ken Rickart.   Ken works for Morris Communications, a very stodgy, family run corporation.  Drupal adoption at Morris would seem a long shot for the traditionalist culture of this media giant.  But with the aid of Ken's obvious leadership and technical skills, Drupal is shaking things up.  If Ken's experience at Morris is any indication, I'd expect we'll hear about more tremors rippling through vaunted institutions and enduring companies triggered by Drupal's "time to market", low cost advantage.  Ken talked about how he rapidly delivered some high value business services to internal users (contract renewal reports) and external media consumers (online editions of local newspapers) with Drupal, demonstrating just how true the "good enough" axiom can be for certain classes of problems, and why that mattered to the big cheese at Morris whose main functions are to manage the bottom line and shake hands with the pros at Augusta National).
I'd better get some sleep.

September 19, 2007 in Web/Tech |

| | |

CommunityNext 2

Start Up style enthusiasm was in no short supply at the second CommunityNext conference held at the Plug And Play Tech Center in Sunnyvale last Saturday.  Several companies represented were not even online as of the first CommunityNext conference held last February, yet many have since built thriving communities with millions of users.  The secret to their success, and the theme of this gathering, was viral marketing.

While February's day long event featured lessons on How to Tap The Wisdom of Crowds, Saturday's teachings might best be described as Getting Inside The Teenage Brain.  The scope of possibilities seemed to have devolved in the intervening six months to a level more concerned with how 14 year old girls will place a widget on their MySpace page than how the network effect can improve the lives of millions.  I left the event feeling like the social networking party had moved to the trailer park and the Anchor Steam on Draught had been supplanted by Pabst in cans.   But I don't spend much time on MySpace, so take my sentiments with a grain of salt (and a lime).

Community Next badge

Amid the inanity of Profile Bling and Breakup Alert best practices there were sensible exchanges about building net communities the viral way.  Some insights from the viral front lines :

  • Metrics matter - measuring effectiveness against goals are as important to running a widget based viral marketing campaign as any Madison Avenue ad campaign.  
  • It turns out that wikis are for everyone - given a wiki as easy as making a peanut butter sandwich, teachers will use it to develop curriculum and engage their classes, as PBwiki discovered.
  • The Facebook platform was deemed by many widget developers here as the API sine qua non.

Perhaps most relevant to a field architect like me was the implicit adoption pattern threaded throughout the presentations: widgets get combined with other widgets to make new and interesting platforms that are essentially loosely coupled composites of fine grained apps.  This seemingly chaotic trend toward Widget-dom foreshadows an adoption pattern that runs orthogonal to the SOA model so many enterprises are pursuing, where heavy duty governance is critical and service discovery with its attendant infrastructure represent costly overhead.  Corporations are spending millions planning multi-year SOA initiatives.  Meanwhile the Facebook platform allows developers to build composite applications quickly, all the governance is essentially embedded in the client libraries, and services are discovered virally - registries and WSDL are not critical to the ecology of a Widget World.  Granted, student social calendars and virtual food fights are a far cry from a CRM/ERP/BI mashup, and identity management and access control through a RockYou widget would make any CIO cringe, but with the addition of JSON support to platforms like Facebook I expect we'll see more complex integrations emerge soon, and WS-XACML shows promise for protecting data exchange between loosely coupled apps according to some rich policy.  How quickly will this hosted RESTful approach displace enterprise owned and operated SOA infrastructure is hard to predict.  No doubt the transformation is driven by many of the same factors driving the redshift market transformation Sun is betting on.

The explosive growth experienced by many of the companies at CommunityNext reaffirms Sun's focus on designing for network services at scale.  The results of that focus, such as Project Blackbox, the Niagra processor, and the Sun Grid, ought to figure heavily in the future of many of these start ups.  Sun's SOA technology, which looks more and more RESTful by the day, could be the best bridge for Enterprise IT to cross into Widget-dom, and a good platform for social networking platforms to adopt in order to penetrate the Enterprise market.

My cryptic and incomplete notes from this Sun sponsored event are below.

July 19, 2007 in Web/Tech |

| | |

OAN Set Up - Just In Time CoolStack Part 2: MySQL

The Coolstack 1.1 AMP package installs the 32-bit version of MySQL by default.  We want to let the horses out of the corral on this SunFire X2200 M2, so we also install the 64-bit version, which is provided as a separate package.  Since we need the 32-bit version in order to compile php5, we keep it in its original  /opt/coolstack/mysql_32bit  location.

After running  mysql_install_db and the other steps in  /opt/coolstack/mysqlREADME we then to prep MySQL to be a first class citizen on Solaris 10.

Convert MySQL to SMF

Like the the CoolStack 1.1 Apache, CoolStack 1.1 MySQL is not integrated with SMF.    Here are the resulting manifest and method files to get MySQL working cleanly as a service:

/var/svc/manifest/network/mysql.xml : 

<?xml version='1.0'?>
 <!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<!--
        Manifest for MySQL
 -->

 <service_bundle type='manifest' name='CSKmysql:mysql'>

 <service
         name='network/mysql'
         type='service'
         version='1'>
         <create_default_instance enabled='false' />
         <single_instance />

       <!--
                  Wait for network interfaces to be initialized.
        -->
                <dependency name='network'
                    grouping='require_all'
                    restart_on='error'
                    type='service'>
                    <service_fmri value='svc:/milestone/network:default'/>
                </dependency>

                <!--
                  Wait for all local filesystems to be mounted.
                -->
                <dependency name='filesystem-local'
                    grouping='require_all'
                    restart_on='none'
                    type='service'>
                    <service_fmri
                        value='svc:/system/filesystem/local:default'/>
                </dependency>

         <exec_method
                 type='method'
                 name='start'
                 exec='/lib/svc/method/CSKmysql start'
                 timeout_seconds='60'>
         </exec_method>

         <exec_method
                 type='method'
                 name='stop'
                 exec='/lib/svc/method/CSKmysql stop'
                 timeout_seconds='60'>
         </exec_method>

         <exec_method
                 type='method'
                 name='restart'
                 exec='/lib/svc/method/CSKmysql restart'
                 timeout_seconds='60'>
         </exec_method>

 </service>
 </service_bundle> 

/lib/svc/method/CSKmysql :

#!/usr/bin/sh
#
#        Method file for MySQL
#
# This uses the MySQL packages from CoolStack 1.1
# CSKmysql
#
# Modify accordingly!
#
# NOTE: Make sure DB_DIR is owned BY the mysql user and group and chmod
# 700.
#

. /lib/svc/share/smf_include.sh

DB_DIR=/site-data0/data
PIDFILE=${DB_DIR}/`/usr/bin/uname -n`.pid

case "$1" in
        start)
        /opt/coolstack/mysql/bin/mysqld_safe --user=mysql --datadir=${DB_DIR} --pid-file=${PIDFILE} > /dev/null &
                ;;
        stop)
                if [ -f ${PIDFILE} ]; then
                /usr/bin/pkill mysqld_safe >/dev/null 2>&1
                /usr/bin/kill `cat ${PIDFILE}` > /dev/null 2>&1 && echo -n ' mysqld'
                fi
                ;;
'restart')
        stop
    while pgrep mysqld > /dev/null
      do
      sleep 1
    done
        start
        ;;
        *)
                echo ""
                echo "Usage: `basename $0` { start | stop | restart }"
                echo ""
                exit 64
                ;;
esac

Then import the service:

# svccfg import /var/svc/manifest/network/mysql.xml

Before starting MySQL we need to put a config file in  /etc.  The example small config provided with MySQL is just right for now.

# cp /opt/coolstack/mysql/share/mysql/my-small.cnf /etc/my.cnf

Then change the datadir setting in /etc/my.cnf to point to the 1.3TB zfs pool on the external StorageTek 3511:

39  datadir=/site-data0/data

Finally, set the data dir with proper ownership:>

# chown myqsql:mysql /site-data0/data

And make sure it starts:

# svcadm -v enable mysql

At this point we're ready to set up the content management system, Drupal.

Additional tips for MySQL on Solaris:

March 06, 2007 in Web/Tech |

| | |

OAN Set Up - Just In Time CoolStack Part 1: Apache

 

The new Solaris AMP stack, a.k.a. CoolStack 1.1 is here. And not a moment too soon, as I sit down to build another server for the Open Architecture Network. This is server #2, which will provide the n+1 scaling and redundancy necessary to keep the the OAN up and functional in the face of any one component failure and through a good slash-dotting. 

Of all the goodies in this new release, it was the GD library that we needed in particular. It is also nice to see suhosin from the hardened-php project included in this release. Here's a quick breakdown of version differences between CoolStack 1.0 and 1.1:


CoolStack 1.0.2
CoolStack 1.1
Apache
2.0.58
2.2.3
PHP
5.1.4
5.2.0
MySQL
5.0.22
5.0.33
install dir
/usr/local
/opt/coolstack

Convert CoolStack Apache to SMF 

First, I notice that the services in coolstack are not integrated with SMF. We need apache to run under SMF so its privileges can be easily limited.  I convert it to SMF, and prepare it for limited privileges by creating a service manifest and service method based on the original apache service shipped with Solaris 10.

# cp /lib/svc/method/http-apache2 \
/lib/svc/method/http-CSKapache2
# cp /var/svc/manifest/network/http-apache2.xml \
/var/svc/manifest/network/http-CSKapache2.xml

In /lib/svc/method/http-CSKapache2 change

 
    11  APACHE_HOME=/usr/apache2
    12  CONF_FILE=/etc/apache2/httpd.conf
    13  PIDFILE=/var/run/apache2/httpd.pid


    20          /bin/mkdir -p /var/run/apache2 

to 

    11  APACHE_HOME=/opt/coolstack/apache2
    12  CONF_FILE=/opt/coolstack/apache2/conf/httpd.conf
    13  PIDFILE=/var/apache2/run/httpd.pid


    20          /bin/mkdir -p /var/apache2/run
 

In /var/svc/manifest/network/http-CSKapache2.xml change 

    10  <service_bundle type='manifest' name='SUNWapch2r:apache'>

    23          <instance name='apache2' enabled='false'>

   100                                  manpath='/usr/apache2/man' />

to 

    10  <service_bundle type='manifest' name='CSKapch2r:apache'>

    23          <instance name='CSKapache2' enabled='false'>

   100                                  manpath='/opt/coolstack/apache2/man' />

Then import the service:
# svccfg -v import /var/svc/manifest/network/http-CSKapache2.xml

Minimize Apache's Service Privileges

Next, we configure the new service to run with minimal privileges following the example in 
Glenn's Limiting Service Privileges BluePrint.  After the procedure the CSKapache2 privileges
should look like this:
# svcprop -v -p start CSKapache2
start/timeout_seconds count 60
start/type astring method
start/exec astring /lib/svc/method/http-CSKapache2\ start
start/user astring webservd
start/group astring webservd
start/privileges astring basic,!proc_session,!proc_info,!file_link_any,net_privaddr
start/limit_privileges astring :default
start/use_profile boolean false
start/supp_groups astring :default
start/working_directory astring :default
start/project astring :default
start/resource_pool astring :default

Note that the changes to the PidFile and LockFile directives specified in this minimization procedure will be overridden  by the Server-pool management configuration that is loaded by 

474  Include conf/extra/httpd-mpm.conf

Unless the corresponding directives are commented out of  /opt/coolstack/apache2/conf/extra/httpd-mpm.conf

Increase Semaphores for PHP 

By default the php5_module is loaded in the CoolStack 1.1 apache.  I observed that PHP was causing the maximum number of semaphores to be exceeded, so I created a project  httpd.php  to bump the max from 128 up to 256:

# projadd -c "Apache-PHP" -U webservd httpd.php
# projmod -sK "project.max-sem-ids=(privileged,256,deny)" httpd.php

then added the project to the service configuration:

svccfg -s http:CSKapache2 setprop start/project = astring: httpd.php

Enable suhosin

Because the site is expected to receive lots of publicity, and it will not have a 24x7 SWAT team ready to jump in and thwart the bad guys, we want it to be as hardened to attacks as possible.  Suhosin gets us a long way toward that goal.  Since it's already built for us in CoolStack, we just need to enable it by  uncommenting  extension="suhosin.so"  in  /opt/coolstack/php5/lib/php.ini


Now we're ready to setup the CoolStack 1.1 MySQL ...


Additional SMF resources:

 

March 05, 2007 in Web/Tech |

| | |

CommunityNext - a Glimpse into the Future

<p><p><p><p><p><p><p><p><p><p>Community Next, Saturday February 10, 2007,  Stanford University</p></p></p></p></p></p></p></p></p></p>

If the folks present at the CommunityNext conference last weekend have anything to say about it, the wisdom of crowds will dominate our future.  This emerging industry alternately known as Social Networking and Online Communities,  has produced some of the most compelling modes of interaction available to society.  And we're only just beginning to understand the significance of, what can only be called, a major social movement.

Sun Microsystems, who sponsored the event, is perhaps the most visible and largest commercial enterprise to declare its support and integral role in advancing the build out of Social Networking.  Sun anticipated the importance of this movement to its business model by proffering that the Age of Participation is upon us and the build out of IT infrastructure to make it possible will be led by the companies that operate as online communities and understand what it means to belong to the network.

Here are my telegraphic notes from the event:

Community Next, Saturday February 10, 2007,  Stanford University

Organizer: Noah Kagan

  • ~ Brand Utopia
    • Rules
      • Change the world
      • Do something you care about
      • Do something worth talking about
    • "I wanna be part of the carbon neutral network"
      • "How do I get credit for using efficient servers?"
  • ~ Citizen Agency, Tara Hunt
    • Rules of enagement
      • community != marketing strategy
        • cluetrain
      • Think like a customer
      • 10 ways to shift your thinking
        • 1. become a community evangelist
          • (reverse of corporate evangelist)
        • 2. shift your measures of 'success'
          • e.g., GPI (not GDP)
        • 3. embrace the chaos
          • e.g., barcamp
        • 4. find your higher purpose
        • 5. understand who you're building that app for
        • 6. INreach, not outreach
          • e.g., twitter
        • 7. design to delight
        • 8. be part of the community you serve
        • 9. have respect
        • 10. have patience
  • ~ DIY Tools, Rohit Bhargava
  • Say Now
    • Voice messages
      • Artist to Fan
      • Fan to artist
      • Fan to fan
  • mPulse media
    • mobile social networking
  • ~ loopt
    • social mapping (geographic)
      • most common text msg: "W U @?"
      • "enhancing serendipity"
      • enabled by location APIs and security, privacy problems solved
      • launched on boost mobile
      • 6 invites per user
      • $2.99/mo.
  • social community demos
  • ~ dogster
    • community guidelines
      • define early
      • enforce them
      • cite them when enforcing
    • core components
      • entertainment
      • information
      • sociality
        • e.g., give a bone
      • services
    • stay w/in your impact horizon
      • don't look too far out
    • tips
      • start w/ less rules, apply more as needed
      • be evolutionary - get feedback
      • skip version releases, favor small responsive changes
    • revenue streams
      • sponsors
        • get the first one!
        • you're never too small
        • can provide gowth capital
        • caution! sponsors don't scale well
      • direct ad buys
      • premium memberships
      • bulk ads
    • circle of trust: dogster-community-advertisers
    • brand campaign
      • be picky - find the right advertisers
      • build campaigns that bring advertiser intot he community
      • don't be deceptive, ever!
      • write your advertiser's copy
      • think beyond banners, find the right place to position their message
    • creative commons attribution
  • ~ skinnyCorp, Jeffrey Kalmikoff & Jake Nickel (Chicago)
    • Four Commandments
      • allow your content to be created by its community
      • put your project in the hands of its community
      • let your community grow itself
      • reward the community that makes your project possible
    • projects
      • naked & angry
        • fashion based on tile patterns upload
      • Extra Tasty
        • drink recipes
        • example of "Your Project is Not Good Enough"
      • poopface
      • yayhooray
      • threadless - Nude No More
      • ~ threadless
        • sold > 1M tshirts
    • raised $100K for Katrina via tshirt designed by employee
  • ~ big in japan, Jake McKee
    • support vs. monetization
    • everybody goes home happy
      • 1. redefine success
      • 2. share.  a lot
      • 4. skip NDA
      • 5. set & maintain expectations
      • 6. train your collegues
  • ~ indieclick, Heather Luttrell
    • top ten myths
      • 1. if we build it they will come
      • 2. members won't cost us anything
      • 3. bandwidth is cheap and won't cost anything more as we grow
      • 4. we can add ad space to the design after we have built and audience
      • 5. our audience won't accept ads
      • 6. ads will look terrible
      • 7. direct sales (or indirect) will sell all our ads
      • 8. we need detailed data to manage ad sales
      • 9. we are getting thousands of hits each day, surely they are worth some ad dollars
      • 10. advertising is our only source of revenue
    • lessons learned

      • designate a community manager

        create an area for advertising discussion

        seek advertisers in your niche

        be willing to customer for advertisers

        con't insist on the highest CPMs

        be lexible - timeframes may slip, creative implementation may be complicated and campaigns may be cancelled

        expect to have advertisers closely monitor delivery

        • fresh content drives traffic
        • newsletters allow you to drive traffic weekly
        • you can fund a mktg campaign from ad revenue
    • 2007 trends
      • tier 1 advertisers getting involved in niche sites
      • non-traditional advertiseing (custom implementations/sponsorships)
      • featured content/video/editorial?
      • UGC and advertising (the consumer-created commercial
      • rich media e.g., expandable banners
      • video
  • ~ claimID, Fred Stutzman
    • Key Concepts
      • privacy is weird
        • 19% sharing with friends
        • 27% sharing with faculty
        • 29% sharing staff
        • privacy is changing
      • utility is important
        • situational relevance
          • many networks - primary and secondary
          • networks are constantly in flux
          • move in to new networks for actual information needs
          • facebook, last.fm (singles bar metaphor: audiences move on, but biz survives)
        • social objects
          • flickr
            • cats in sinks
          • del.icio.us
        • objects provide a shared solcial experience
      • value is granular
        • see metcalfe's law
        • social communities offer a rich experience - more than a binary
        • understand value creation - what's initial value vs. how much value is added by the community?
    • Takeaways
      • respect privacy expectations
      • niche into situational relevance
      • leverage social objects
      • understand the network's multiplier

February 13, 2007 in Web/Tech | | Comments (3)

| | |