July 02, 2009

Dave MinerOpenSolaris Power User Tutorial at OSCON

July 02, 2009 06:04 PM GMT
I really should have posted this quite some time ago, but between getting the OpenSolaris 2009.06 release out, speaking at CommunityOne, speaking at the OpenSolaris user group in New York, and trying to sleep once in a while, it's been a little tough to keep up.  Anyway, Nick and I are giving a three-hour OpenSolaris tutorial at OSCON 2009 on July 21. Looking at the content draft, we've probably got more like five hours of material, but we'll figure out how to cram most of it in.  Even if you've read OpenSolaris Bible you're likely to learn a lot, as a fair amount of the material is on technology that's not covered in the book, such as Crossbow and the Automated Installer.  I'm also expecting to spend some time wandering around at the conference, so hope to see you there!


OSCON 2009


July 01, 2009

John GardnerNetbeans version 6.7

July 01, 2009 06:33 AM GMT
Always the way, no sooner do I get round to finally upgrading to Netbeans v6.5.1, then out comes v6.7. Once I pluck up the energy to upgrade again, I'll write what the new features etc are like.

Tom HaynesNew BFU bits and closed-bins for closedv11

July 01, 2009 06:01 AM GMT

I just did a branch merge of the nfs41-gate with the onnv_117 tag of the onnv-gate.

And the closed tree had some changes.

I've added a tag of closedv11 to the nfs41-gate and you can download new versions of the closed-bins at http://www.opensolaris.org/os/project/nfsv41/downloads/


There are two ways you can go about installing pNFS on your machines:

  1. Get an install media for the base release (i.e., 117 in this case), install it, and then use the BFU (think precompiled kernel) and SUNWonbld supplied tarballs to install our pNFS image on top of the base. Details for doing that can be found at the bottom of http://wikis.sun.com/display/NFS/July+2008+Release+Notes
  2. Retrieve either a copy of our public Mercurial gate (ssh://anon@hg.opensolaris.org/hg/nfsv41/nfs41-gate) or the source tarball on the Downloads page. Follow the instructions at http://opensolaris.org/os/downloads/on/ on how to build ON. We've provided the source and the closed binaries, you supply the compile cycles.

Originally posted on Kool Aid Served Daily
Copyright (C) 2009, Kool Aid Served Daily

June 30, 2009

Marcelo LealFMA Messages WIKI way…

June 30, 2009 11:48 PM GMT
Almost perfect!! Wow, that was a really, really nice surprise! I do like all the new stuff from Solaris 10/OpenSolaris, and i say that in every opportunity, like now. I did write some posts about FMA, like this one… but one problem about the fmdump messages IMHO, was the fact that the messages on console were [...]


Jim Grisanzio1 Minute at CommunityOne

June 30, 2009 01:10 PM GMT
Ashwin Bhat and Angad Singh asked me for one minute of my time outside the keynote hall at CommunityOne a few weeks ago. Hey, what's a minute, right? Happy to. But this minute was to be digitally recorded. Uha. Video. I generally shy away from such things because I`m shy about being interviewed. But these are really good guys and they`ve done great work as Campus Ambassadors in India, so I felt safe in front of their camera (though I`d clearly much rather be behind the camera). It wasn't too bad, though. But the 1 minute ran for 2 minutes and 21 seconds! Anyway. Thanks, guys. Great fun. Hope to get back to India sometime soon.

June 29, 2009

Jim GrisanzioJapan OpenSolaris Community Meeting 062709

June 29, 2009 02:44 PM GMT

The Japan OpenSolaris Community together on Saturday. Nice day (and night). About 60 people came by for the three sessions, two of which were in Japanese and the third in English. Then all three groups came together for a nomikai. I think the model works well to start integrating the Japanese and international OpenSolaris communities.

I used a new lens for this event. My f/1.4 lens is getting fixed, so I borrowed Jon`s 50mm f/1.2, which is one scary smart lens. It`s a tad expensive, too, so I was more than a little nervous shooting with it. Anyway, at f/1.2 the focus is just razor thin. Focus on someone`s glasses and their entire face is out. I messed up a few images that way, but by the end of the night I was getting used to it. Amazing piece of glass. By the way, you can see Jon`s stuff here. He`s one of the best photographers around.

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OSUG 062709 Japan OSUG 062709

Japan OpenSolaris Community: http://blogs.sun.com/jimgris/tags/jposug
Michael Sullivan presentation on OpenSolaris & Storage: OSol + ZFS = NAS

Simon Phippslinks for 2009-06-29

June 29, 2009 02:30 PM GMT

Jim GrisanzioTokyo Hackerspace

June 29, 2009 08:20 AM GMT
I've been checking out the Tokyo Hackerspace gmail list for a few weeks. Looks very interesting. The project grew out of some discussions at BarCamp Tokyo a couple of months ago, and I spoke to Karamoon about it at the OpenSolaris community event this weekend. In a world of ever expanding global digital communities, it seems like a nice idea to have a very local a very physical space to hang out in and hack on things that need hacking. Global and digital are fine, but local and physical are needed too. For info, check it out on the wiki.

Jim GrisanzioWebsite Transition: Two Phase 1 Documents Posted

June 29, 2009 02:35 AM GMT
This morning Bonnie posted two documents supporting the Phase 1 website transition plans:
  1. The plan to implement the governance and website roles and collectives, and
  2. The data migration strategy outlining how data will be migrated from existing databases into the new Auth database.
Bonnie and Alan drafted the documents and all three of us iterated for a couple of weeks as they went through multiple drafts. It`s amazing experiencing the distinction between writing a document that articulates some issue in theory and writing a document that articulates a specific implementation that has to actually work. It`s the distinction between night and day. Ideas are fine, but if you don`t build them they are not real. That lesson is learned.

Also, I appreciate more than ever the process I went through in the recent past attempting to re-write the OpenSolaris Constitution. Building and describing the new site would have been so much easier had that Constitution been approved in March. But it wasn`t. That`s life I suppose. So, now we have to implement the old Constitution while also accounting for things that document doesn`t even mention because it came about after the original site was designed. Not to mention all the odd stuff that evolved (and broke) on the current site -- all of which has to be migrated to the new site. As a result, in August we will have some things covered under Governance and some things covered under generally accepted practice -- and that last bit was really the basis of the concept we were trying to move toward with the proposed Constitution. Hopefully, the OGB will at some point this year take up that proposed Constitution again, get it updated, and get it approved so our Governance documents reflect the reality of how the community operates in real life.

Anyway, until that happens we will continue building what we have to build, and it will be good to finally break with the past of the old site. So, it`s important for anyone with an account on opensolaris.org to review these new documents and the other information we have posted in the Website community recently to be prepared for the changes coming in July and August. All users on the site will be affected by this multi-phase transition (hopefully in a good way, of course). More documents will be posted in the coming weeks on website feature mappings, Auth transition instructions, and content migration plans. And that`s just Phase 1 and Phase 2. There will be a Phase 3 that will take us well into the fall.

Website Transition Documentation | Auth System Beta | XWiki Website Beta | Program Roadmap

Jim GrisanzioOpenSolaris PDAs?

June 29, 2009 02:33 AM GMT
"I hope that after some time we'll see OpenSolaris powered PDAs." -- Alexander Eremin

Tom HaynesDebugging NFS4ERR_BADSESSION from a DS

June 29, 2009 02:26 AM GMT

We decided to skip branch merges to Nevada builds 113-116. We had issues with the hardware in the lab and vboxes. Plus we wanted a stable environment for the BakeAThon. I'm having a hard time merging with build 117. One of the issues was a panic on the client (which I could patch) due to getting NFS4ERR_BADSESSION from a DS. I was able to see the DS replying on the exchange_id request in a packet trace. There are only a handful of places where the server returns this error code, so I was pretty sure I could reuse a Dtrace script I had laying around:

:nfssrv:mds_lease_chk:return
/args[1] != 0/
{
        printf("rc1 = %d", args[1]);
}

Until I came along this code snippet:

                                *cs->statusp = resp->dsr_status =
                                    NFS4ERR_UNSAFE_COMPOUND;
                                goto final;
...
final:
        DTRACE_NFSV4_2(op__destroy__session__done,
            struct compound_state *, cs,
            DESTROY_SESSION4res *, resp);
}

Just great, no return value. How could I catch it?

Well, the answer is staring us right in the face, we can use op__destroy__session__done. I had to snoop around for examples in a colleague's home directory (which is kinda why I blog about this stuff, it is easier to find), but ended up with this:

:::op-bind-conn-to-session-done
{
        bresp = (BIND_CONN_TO_SESSION4res *)arg1;

        printf("rc1 = %d", bresp->bctsr_status);
}

:::op-destroy-session-done
{
        dresp = (DESTROY_SESSION4res *)arg1;

        printf("rc1 = %d", dresp->dsr_status);
}

Note that I wasn't too worried about being specific as to which module you found these calls in, I was betting on them being very unique.

Oh, and I was able to narrow down where the NFS4ERR_BADSESSION was coming from. And then I had to add debug statements to find out why. :-< I bet I might have been able to do it still with Dtrace. :->


Originally posted on Kool Aid Served Daily
Copyright (C) 2009, Kool Aid Served Daily

June 28, 2009

Jim GrisanzioNew ZFS Book Coming to Japan

June 28, 2009 06:50 AM GMT

Four members of the Japan OpenSolaris Community wrote a book on ZFS recently. It`s coming out in July, and it`s specifically for the Japanese market. The cover has not been selected yet, but here are the early details: ZFS 仮想化されたファイルシステムの徹底活用 (大型本) by Hisayoshi Kato, Michitoshi Sato, Nagahara Niroharu, and Imai Satoshi. This is quite a significant contribution to the community in Japan because it`s important to have technical content written by Japanese engineers for Japanese engineers. Translating English content from the west good, of course, but the generation of original content in Japanese also needs to be part of this community`s growth plans.

Here are some more books on OpenSolaris: http://blogs.sun.com/jimgris/tags/book

June 27, 2009

Jim GrisanzioJapan Linux Symposium: Tokyo, October 2009

June 27, 2009 02:53 AM GMT
It's excellent to see the 9th Annual Linux Kernel Summit and the 1st Annual Japan Linux Symposium coming to Tokyo in October. Check out this language from the LF website: "The Japan Linux Symposium will be the showcase Japan and Asia Pacific conference for The Linux Foundation." Showcase. This is significant. The Japanese may not shout about it much, but developers in this country are contributing to FOSS and their contributions are growing. The potential in Japan for open source is huge. I've been saying it since I got here. So cool that the LF clearly recognizes this potential by bringing their conferences here. Also interesting: the LF website appears in two languages -- English and Japanese.



I already hang out with the Tokyo Linux User Group (here, here), so I hope to attend this gig in October.

Jim GrisanzioOpenSolaris in Brazil: Right at the Top!

June 27, 2009 02:04 AM GMT
Nothing like going right to the very top, eh? My goodness. Here's the OpenSolaris community in Brazil at FISL hanging out with Brazil's President Luiz Inácio Lula da Silva. I think this sets a new standard in government relations for the entire OpenSolaris community, don't you think? So, each one of us around the world now has to go out and shoot some images (video is fine, too) of our country's leader standing with our respective communities all dressed up in OpenSolaris stuff. Ok. Should be easy enough. Just send your images to osug-leaders or advocacy-discuss, and we'll collect them there.

Absolutely. Outrageous.

June 26, 2009

Marcelo LealFISL 10

June 26, 2009 11:38 PM GMT
Congratulations to Vitório Sassi and all OSOL users from OpenSolaris-BR (special to PoaOSUG ;-) that are doing a great work at FISL 10!


Jim GrisanzioOpenSolaris b117

June 26, 2009 02:16 PM GMT
I just updated to OpenSolaris development build 117. Easy. Go get it here.

June 25, 2009

Jim GrisanzioShooting the Street in NYC

June 25, 2009 05:20 PM GMT
I love these six videos of street photographers walking around shooting images in New York City. My fav is Sandra, who said, "Wow, this is what a father looks like when no one`s looking." She photographs day laborers. She gets that trust takes time.

Jim GrisanzioEarning Confidence

June 25, 2009 02:51 PM GMT

A Workers’ Paradise Found Off Japan’s Coast: “Mr. Fujimoto said he would resign immediately if a serious rival appeared in an election. `That would be a sign the village has lost confidence in me,` he said.” -- New York Times

Interesting. I really must visit this place.

Ben RockwoodSolaris Automated File Integrity Checking: bartlog

June 25, 2009 08:38 AM GMT

The Solaris Basic Audit & Reporting Tool, bart, is a great little alternative to Tripwire or AIDE. While not nearly so robust or full featured, it does what you need it to do with very little impact. The sqlite of intrusion detection systems, if you will. I blogged about BART in 2005 and so far its still only got 1 real comment, which was simply mentioning AIDE as an alternative. No love.

Given that BART is awesome and no one seems to embrace it due to, perhaps, perceptions of complexity that are unfounded, I sought to implement a simple solution to bring BART to the masses. I call it bartlog

Quite simply, bartlog is a BASH wrapper around BART and logger which is run from cron on any schedule you like and reports any changes to syslog. Setup is simple, download bartlog and copy into /usr/sbin or whereever you prefer, then download bart.rules and copy into /etc. Now run bartlog from cron every hour or day or whatever you like.

The script is simple and intended to be tweeked, modified and made as l337 as you like. What it does is creates a BART manifest (record of files and MD5 checksums) for those directory structures specified in the bart.rules file. The first time it runs it just creates a manifest and exits. The second time you run it it creates a new manifest and then compares it against the previously created one. If it doesn't find any changes it just replaces the old manifest with the new one, this avoids you getting repetitiously alerted. However, if it does find a change it sends the change to syslog, so that its stored with your normal logs viewed either by running dmesg or reading /var/adm/messages. By default I'm using the syslog audit.err priority because by default Solaris sends those messages to /var/adm/messages, however if you are deploying this in a production environment I'd recommend using audit.warn instead and then modifying /etc/syslog.conf to send those warnings to a secure centralized syslog server. If you complete the solution with Splunk you could have a centralized, searchable log of all changes to critical files on which you could report, respond or alert on.

I use a one hour interval on my home workstation. Here's my syslog following a new user addition:

root@quadra ~$ dmesg
...
Jun 25 11:01:58 quadra root: [ID 702911 audit.error] BART Reports Change: /etc/.pwd.lock mtime 4a218d04 4a43b0bd 
Jun 25 11:01:58 quadra root: [ID 702911 audit.error] BART Reports Change: /etc/opasswd size 968 985 mtime 49fa4236 4a218d2b contents fc27c5b28b3a248b6c6129aa9aed7329 2200107fc7128d5cd38de333bea4500f 
Jun 25 11:01:58 quadra root: [ID 702911 audit.error] BART Reports Change: /etc/ouser_attr mtime 4a04a741 4a218d01 
Jun 25 11:01:58 quadra root: [ID 702911 audit.error] BART Reports Change: /etc/passwd size 985 1022 mtime 4a218d2b 4a43b0ac contents 2200107fc7128d5cd38de333bea4500f 640da69537a35046571b4fda1def10d1 
Jun 25 11:01:58 quadra root: [ID 702911 audit.error] BART Reports Change: /etc/shadow size 708 783 mtime 4a218d04 4a43b0bd contents f83158dffddc124dab2f22a979338695 6ba7d42600da8d4fc9b8a92f4bf0afe7 
Jun 25 11:01:58 quadra root: [ID 702911 audit.error] BART Reports Change: /etc/user_attr mtime 4a218d01 4a43b0ac 

So I hope this fills the hole. Anyone running a Solaris system at home can download these two files, add to cron and be off and running. No hassle, no maintenance. All the love, none of the pain. If your running a system where bart isn't installed, just install SUNWbart from IPS or the install media.

Cyril PliskoSXCE build 117 is out

June 25, 2009 03:16 AM GMT
Solaris Express Community Edition (aka Nevada) build 117 has been released.

Significant changes in this build:

1. Solaris Simnet PSARC/2009/200 simulated networks
2. zpool autoexpand property - PSARC/2008/353
3. Areca 7.1 backup suite integrated
4. NVIDIA graphics driver updated to 185.18.14
5. Wireshark updated to version 1.0.8
6. Conman 0.2.4.1 console management tool integrated

Changes on package level:

New packages:
system      SUNWareca           Areca backup utilities
system SUNWconmanr ConMan - Console Management tool (root)
system SUNWconmanu ConMan - Console Management tool (usr)
system SUNWperl-net-ssleay Net::SSLeay- Perl extension for using OpenSSL
system SUNWxsun-headers X Window System: Xsun server SDK headers

June 24, 2009

Marcelo LealHP LeftHand Event

June 24, 2009 11:56 PM GMT
Today i was invited to the HP LeftHand Event here @SP. It’s always nice to see and hear about news on the Storage world, and i really like HP hardware. I do remember the first Windows Servers i did manage, all running on HP hardware, ECC memories… old times. No hardware problems, and a lot [...]


Chris MorganEverybody's at it now...

June 24, 2009 11:17 PM GMT
At last, there is a Solaris blog aggregator - Planet Solaris the perfect place to feast on all things Solaris from such luminaries as Casper Dik, Jeff Bonwick and other Sun employees and of course, the non-Sun-rabble such as Rich Teer, Ben Rockwood. I'd add "and me" if I'd done any meaningful Solaris blogging yet...

Darren Moffatprintf should not SEGV when passed NULL for %s format

June 24, 2009 05:14 PM GMT
Author: Darren Moffat 
Repository: /export/onnv-gate
Total changesets: 1

Changeset: acbef346fd18

Comments:
PSARC/2008/403 libc printf behaviour for NULL string
6724478 libc printf should not SEGV when passed NULL for %s format

Files:
modified:
	usr/src/lib/libc/port/print/doprnt.c

 

Finally got this in.

Jim GrisanzioOpenSolaris 2009.06 Launch: Tokyo

June 24, 2009 04:05 AM GMT

I went to the Tokyo launch of OpenSolaris 2009.06 at Sun's office in Jingumae earlier tonight. Good turn out of about 100 people. The lineup: Introduction, Akira Ohsone; OpenSolaris 2009.06, Shunsuke Kuroda; OpenSolaris Demo, Shoji Haraguchi; Solaris 10 5/09, Hiroaki Nozaki; and CommunityOne West Report, Masafumi Ohta. See videos here from Shoji. This is the third such launch of OpenSolaris in Tokyo. Both the 2008.05 and 2008.11 events were excellent as well. Tonight seemed to be an interesting mix of Sun Solaris 10 customers and the growing OpenSolaris communities in Tokyo. There was also some chatter on #opensolaris-jp on IRC. Nice night.

OpenSolaris 2009.06 Launch Tokyo OpenSolaris 2009.06 Launch Tokyo

OpenSolaris 2009.06 Launch Tokyo OpenSolaris 2009.06 Launch Tokyo

OpenSolaris 2009.06 Launch Tokyo OpenSolaris 2009.06 Launch Tokyo

OpenSolaris 2009.06 Launch Tokyo OpenSolaris 2009.06 Launch Tokyo

OpenSolaris 2009.06 Launch Tokyo OpenSolaris 2009.06 Launch Tokyo

OpenSolaris 2009.06 Launch Tokyo OpenSolaris 2009.06 Launch Tokyo

OpenSolaris 2009.06 Launch Tokyo OpenSolaris 2009.06 Launch Tokyo

OpenSolaris 2009.06 Launch Tokyo OpenSolaris 2009.06 Launch Tokyo

OpenSolaris 2009.06 Launch Tokyo OpenSolaris 2009.06 Launch Tokyo

OpenSolaris 2009.06 Launch Tokyo OpenSolaris 2009.06 Launch Tokyo

OpenSolaris in Japan: http://blogs.sun.com/jimgris/tags/jposug
OpenSolaris Launches in Japan: http://blogs.sun.com/jimgris/tags/os-tokyo-launches

June 23, 2009

Chris MorganSolaris 10 kicks ass

June 23, 2009 03:17 PM GMT

I've been struggling to get my webcam to work with my Apple Powerbook. I would have thought it had enough performance - Nvidia graphics, fast ethernet, 867 Mhz PowerPC G4 etc etc. For some reason it just can't show the full-size streaming video.

Running the same feed into my Sun Blade 1000 with Solaris 10 and I get a rock-solid 640x480 video feed in my browser.

Both laptop and workstation are plumbed into a switch, which the webcam is also connected to. The Sun machine just hooks up and works. The Apple simply struggles. I've tried using a direct cable (crossover, and non-crossover) into the laptop. I tried fiddling with the ethernet parameters.

Maybe it's because the workstation has 2 cpus, or something to do with hardware, but it could just be Solaris 10 kicks ass.

Jim GrisanzioExtraction

June 23, 2009 03:16 PM GMT
Communicating is all about building relationships, and that`s always a two way street (or if you are in the community business, a multi-way street). Every wonder what a rapid fire relationship with Rahm is like. Check out Ring, ring, it's Rahm:
NBC’s Chuck Todd calls the Emanuel relationship “no-nonsense.”

“He’s always trying to extract as much information as he’s trying to give,” Todd says.

But the conversations with Emanuel “can be as little as 30 seconds,” Paul Begala, the CNN commentator and longtime Emanuel friend says. “He calls, drops a few F-bombs, makes his point and hangs up.”
The shock value of his delivery is interesting (he can do that because he`s powerful), but even more important is the bit about the information extraction. Information has to flow both ways to demonstrate the value of the relationship.

Chris MorganYAWAS (Yet another weblog about Solaris)

June 23, 2009 02:17 PM GMT

Seems like everyone's doing it, so I thought I'd do it too.

I am a member of the Open Solaris pilot program, which is a precursor to the full public launch of Solaris as an open source project.

I'm a big fan of all things Unix, having used various Solaris, Linux and Mac OS X systems over the years. I have a particular fondness for Solaris having been working with it for about 10 years now, mostly in the area of GUI programming. Currently I work for a financial data firm in Manhattan.

I was a technical reviewer of "Solaris Systems Programming" by Rich Teer which was just published this year. See here for Rich's home page and a link to buy the book.

Chris MorganWhich would you rather do?

June 23, 2009 08:17 AM GMT

The choices were to watch "CSI" about a flight attendant brutally stabbed to death in a hotel room, or fit my DDS3 drive and get my workstation backed up.

It's obvious, right? I fitted the tape drive. It actually took about 10-15 minutes of real work, plus 1/2 hour searching for a power cable extender (you know, those internal power cables with molex connectors? I have a bunch of extenders somewhere). I never found the extender, so I snipped a cable tidy holding back the existing one inside the box and got the tape drive fully hooked up.

I think a SB1000 looks very nice with a DVD-ROM and a tape drive. Sure it would be even better to have a DDS4 or DAT 74 or whatever the latest kind is, but they cost about 5X more!

As I write this, I'm doing a level0 backup of the entire machine to tape. Quick, are there files at risk where you (yes, YOU!) live? :)

To get things kicked off, I'm using hostdump.sh which takes 2 minutes to download install and works out what to dump all by itself.

Yes, I know that real men write their own scripts, using star, fancy block sizes, FIFO buffers etc, but I've always thought it better to start with protecting the data, then work on the sophistication and efficiency of the backup system strictly in that order.

Chris MorganThe SPARC list

June 23, 2009 06:17 AM GMT

I need to rationalise my SPARC machines, so where better to start than with an inventory? I need to see if I can get rid of some of these, because we have to move house soon, and right now we have a big basement. I can't count on having that much room going forward.

June 22, 2009

Tom HaynesTrip report - BakeAThon at NetApp

June 22, 2009 07:27 PM GMT

So we had our first BakeAThon hosted by NetApp. I thought it went very well and it enabled the NetApp folks to see how much work goes into hosting the event.

I can't go into details that involves other vendors, because of a NDA, but I can say that we did a lot of testing on new features in both the client and the server. I'm not that familiar with the client side changes, I think we had the compound constructor changes by Bob Mastors and Karen Rochford was testing a rewrite of the layout handling code.

I was more excited about the changes on the server side:

I also brought a major rewrite of the layout handling code (allowing for multiple layouts, etc), device handling (allowing multiple datasets on the same DS) and the integration of the kspe (kernel simple policy engine), but I probably needed another week of development/testing on that code.

The other major development is that we are finally making the switch to using Virtual Boxes for our testing needs. We typically would want 3-4 machines per developer. We can refine this by having a pair of public communities (MDS plus 2 DSes). One as the stable system and the other as a sanity test rig. And then each client developer gets their own machine for a client and each server developer gets a community.

Well, with Virtual Boxes working on a wide range of host OSes, we can have the communities hosted on one or two beefier machines, perhaps sharing room with the build server, and each developer can bring up what they need on their laptops.

I'm going to be giving a presentation on July 7th over pNFS at the Oklahoma City OpenSolaris User Group - OKCOSUG. I'd like to give a live demo using virtual boxes, but I'm not making any promises...


Originally posted on Kool Aid Served Daily
Copyright (C) 2009, Kool Aid Served Daily

Robert MilkowskiOpenSolaris Apps of Steel Challenge

June 22, 2009 01:01 PM GMT
Recently I participated in OpenSolaris Apps of Steel Challenge and I won a laptop!
My Toshiba Portégé® R600 arrived this morning and it comes pre-installed with Open Solaris. First impressions are really good - it is so light.

It is a fortunate coincidence that I'm on a short holiday right now as I will have more time to play with it :) Already doing an upgrade.

Thank you Sun you've made my day!

Alan HargreavesLive Upgrade and TimeSlider gotcha

June 22, 2009 01:05 AM GMT

Tried to upgrade my workstation over the weekend to snv_117. Apart from a little tridying up I had to do as a package didn't install correctly, all apeared to be going fine. I then went to unmount /.alt.snv_117, and it failed saying that the filesystem was busy.

fuser -c showed no processes using the mount point. What could it be?

A little bit of dtracing the umount2() system call was illuminating.

  1              <- zfsctl_umount_snapshots           0                0
  1            <- zfs_umount                          0               16

Hang on, snapshots? Although it returned 0, let's just check; as I do have timeslider enabled on this box.

rootksh@vesvi:~$ zfs list -t snapshot|grep 117                                                                     
pool/ROOT/snv_116@snv_117                                   4.03M      -  8.78G  -
pool/ROOT/snv_117@zfs-auto-snap:hourly-2009-06-19-09:00     43.8M      -  7.99G  -
pool/ROOT/snv_117@zfs-auto-snap:hourly-2009-06-19-10:00     48.9M      -  8.44G  -
pool/ROOT/snv_117@zfs-auto-snap:hourly-2009-06-19-11:00     43.7M      -  8.74G  -
pool/ROOT/snv_117@zfs-auto-snap:frequent-2009-06-19-11:15   42.6M      -  8.75G  -
pool/ROOT/snv_117@zfs-auto-snap:frequent-2009-06-19-11:30   45.8M      -  8.76G  -
pool/ROOT/snv_117@zfs-auto-snap:frequent-2009-06-19-11:45   38.1M      -  8.77G  -
pool/ROOT/snv_117@zfs-auto-snap:hourly-2009-06-19-12:00     38.5M      -  8.80G  -
pool/ROOT/snv_117@zfs-auto-snap:daily-2009-06-22-00:00          0      -  8.80G  -
pool/ROOT/snv_117@zfs-auto-snap:weekly-2009-06-22-00:00         0      -  8.80G  -
pool/ROOT/snv_117@zfs-auto-snap:hourly-2009-06-22-10:00         0      -  8.80G  -
pool/ROOT/snv_117@zfs-auto-snap:frequent-2009-06-22-10:30       0      -  8.80G  -
pool/ROOT/snv_117@zfs-auto-snap:frequent-2009-06-22-10:45       0      -  8.80G  -

Oh, timeslider was taking snapshots of the filesystem while it was upgrading. Hmm maybe we should be having that disabled on the target of a live upgrade (rfe coming, but I don't hold out a lot of hope).

Anyway, removing them was not difficult:

rootksh@vesvi:~$ zfs list -t snapshot|grep snv_117@zfs-auto|awk '{print $1}' | xargs -L 1 zfs destroy
rootksh@vesvi:~$ luumount snv_117                                                                                  
rootksh@vesvi:~$ 

Something to keep in mind if you are using timeslider, zfs root and live upgrade (I wonder if we would have the same issue with 'pkg image-update' in OpenSolaris).

June 21, 2009

James DickensHappy Fathers Day

June 21, 2009 09:19 PM GMT
Happy Father day to all the Fathers out there. Have a great day.

Jim GrisanzioDefend Yourself

June 21, 2009 05:20 PM GMT
When I was in San Francisco a few weeks ago, I picked The 48 Laws of Power and The 33 Strategies of War by Robert Greene. If you love history and the study of how things really get done, check these books out. I can`t put them down. I can see how Greene`s stuff would come handy while defending yourself against the packs of wild dogs running around out there. Some very nice tips in these books. Greene seems like an interesting character, too. Study hard. I am.

June 20, 2009

Jim GrisanzioPink Backhoes with Attitude

June 20, 2009 03:12 AM GMT
I love this. An excavating contractor has five or six heavy backhoes on this site ripping apart an old concrete building. And they are painted a lovely shade of pink with white polka dots. How`s that for business development for an industry not used to such style, eh? I was on a bus driving by the other day and had my little camera, so I really couldn`t get a good shot, but you get the idea. This is a contractor with an attitude, no question about it. And in Japan, too. Wild. I`d like to meet him. Or perhaps her?

Pink Backhoe Pink Backhoe

June 19, 2009

Robert MilkowskiIntercepting Process Core Dumps

June 19, 2009 03:11 PM GMT
We disabled process core dumps on one of our environments but still we want to know when it happens along with some more information on the even.

root@ dtrace -q -n fbt:genunix:core:entry \
'{printf("%Y exec: %s args: %s cwd: %s pid: %d zone: %s signal: %d\n", \
walltimestamp, curpsinfo->pr_fname, curpsinfo->pr_psargs, cwd, pid, \
zonename, arg0);}' >/local/tmp/process_cores.log

Now lets try to kill a process so it tries to dump a core:

root@ bash -x
root@ kill -SIGBUS $$
+ kill -SIGBUS 14054
Bus Error (core dumped)
root@

root@ tail -1 /local/tmp/process_cores.log
2009 Jun 19 16:07:54 exec: bash args: bash -x cwd: /home/milek pid: 14054 zone: global signal: 10
root@
The overhead of running the script is practically none unless you're trying to dump as many core dumps as possible per second and even then the overhead should be relatively small :)

James Dickensserve weather

June 19, 2009 06:15 AM GMT

its hard to sleep when there is this many bright colors on the weather map, and they are expecting hail.

those in california aren't used to seeing this many colors on the radar map all the colors are usually associated with loud noise from thunder, lightning, and possibly hail, which is expected in the next few minutes.


June 18, 2009

James DickensRoy Wood has a really interesting number

June 18, 2009 08:44 PM GMT
Just read this really interesting blog entry about an interesting number he recived in an email.

Its really cool. Check it out.

Robert MilkowskiCognitive Computing via Synaptronics and Supercomputing (C2S2)

June 18, 2009 12:01 PM GMT
Cognitive Computing via Synaptronics and Supercomputing (C2S2):
"By seeking inspiration from the structure, dynamics, function, and behavior of the brain, the IBM-led cognitive computing research team aims to break the conventional programmable machine paradigm. Ultimately, the team hopes to rival the brain’s low power consumption and small size by using nanoscale devices for synapses and neurons. This technology stands to bring about entirely new computing architectures and programming paradigms. The end goal: ubiquitously deployed computers imbued with a new intelligence that can integrate information from a variety of sensors and sources, deal with ambiguity, respond in a context-dependent way, learn over time and carry out pattern recognition to solve difficult problems based on perception, action and cognition in complex, real-world environments."

Robert MilkowskiSolaris Technologies Introduced for Oracle Database

June 18, 2009 11:54 AM GMT

click here

Roy Wood142857 is an Interesting Number

June 18, 2009 10:45 AM GMT
A couple of days ago a colleague sent me this interesting number, 142857, via email. I performed a Google search on the number and I was surprised to see so many returns - 159,000. It even has its own wiki page. For additional information, here is the wiki page for this cyclic number - 142857.

Below is the email.

Subject: Interesting Numbers

Roy,

I want to let you know a weird number which is 142857.

Why?

142857 X 1 = 142857
142857 X 2 = 285714
142857 X 3 = 428571
142857 X 4 = 571428
142857 X 5 = 714285
142857 X 6 = 857142

See, the answers show all same numbers with different position.

142857 X 7 = 999999

142 + 857 = 999
14 + 28 + 57 = 99

142857 X 142857 = 20408122449
20408 + 122449 = 142857

Interesting, huh?

Jim GrisanzioTokyo CGM Night Episode 5: 061609

June 18, 2009 07:41 AM GMT

Here are some images from Tokyo CGM Night Episode 5 at KDDI Web Communications the other night. The event was hosted on by Danny Choo and Andrew Shuttleworth, of course, and it was nice to see a lot of friends and some interesting new characters as well. Here is the home of CGM night at Danny Choo`s place. Strange night, though. Too much light. Just as I was starting to get used to shooting in the dark they turned the lights on. Great fun as usual, though.

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

Tokyo CGM Night 5 Tokyo CGM Night 5

http://blogs.sun.com/jimgris/tags/cgm
Tokyo CGM Night Pool on Flickr

Jim Grisanzio2 Deadlines to Create New Website Collectives

June 18, 2009 07:32 AM GMT
As part of the new schedule for the website transition, there are two important deadlines to get collective infrastructure (Communities, Projects, User Groups) on opensolaris.org:

June 30th: Hidden Collectives: All collectives in a hidden state on opensolaris.org must be opened by June 30th or they will be deleted. Mail has been sent to these collective owners who will be affected. A few weeks ago there were about 40 hidden collectives, but that list is down to about a dozen now. On June 30th it will be zero. Also, on the new site, there will be no feature for "hidden" collectives. The current site's feature of hiding projects is a badly implemented bug. Any user logged into the site can find a hidden project if they know the URL (or are good at guessing), and repos from hidden projects show up in searchers as well. The new site will not put up with this.

July 20th: New Collectives: There will be a moratorium on setting up new collectives starting July 20th as we prepare to implement Phase 1 of the transition (Auth database goes live) at the end of July. If you have outstanding proposals for new Communities, Projects, or User Groups, you must send your approval threads to the project-setup list by July 20th. Please refer to the new Website Infrastructure: Collective Life Cycle Instructions document for implementation details when requesting new infrastructure.

Robert MilkowskiHappy Birthday

June 18, 2009 06:15 AM GMT
Yesterday's LOSUG was a little bit surreal as we were singing happy birthday to OpenSolaris and had a birthday cake and a champagne.

June 17, 2009

Adam LeventhalSun Storage 7410 space calculator

June 17, 2009 10:41 PM GMT

The Sun Storage 7410 is our expandable storage appliance that can be hooked up to anywhere from one and twelve JBODs with 24 1TB disks. With all those disks we provide the several different options for how to arrange them into your storage pool: double-parity RAID-Z, wide-strip double-parity RAID-Z, mirror, striped, and single-parity RAID-Z with narrow stripes. Each of these options has a different mix of availability, performance, and capacity that are described both in the UI and in the installation documentation. With the wide array of supported configurations, it can be hard to know how much usable space each will support.

To address this, I wrote a python script that presents a hypothetical hardware configuration to an appliance and reports back the available options. We use the logic on the appliance itself to ensure that the results are completely accurate as the same algorithms would be applied as when then the physical pallet of hardware shows up. This, of course, requires you to have an appliance available to query — fortunately, you can run a virtual instance of the appliance on your laptop.

You can download the sizecalc.py here; you'll need python installed on the system where you run it. Note that the script uses XML-RPC to interact with the appliance, and consequently it relies on unstable interfaces that are subject to change. Others are welcome to interact with the appliance at the XML-RPC layer, but note that it's unstable and unsupported. If you're interested in scripting the appliance, take a look at Bryan's recent post. Feel free to post comments here if you have questions, but there's no support for the script, implied, explicit, unofficial or otherwise.

Running the script by itself produces a usage help message:

$ ./sizecalc.py
usage: ./sizecalc.py [ -h <half jbod count> ] <appliance name or address>
    <root password> <jbod count>
Remember that you need a Sun Storage 7000 appliance (even a virtual one) to execute the capacity calculation. In this case, I'll specify a physical appliance running in our lab, and I'll start with a single JBOD (note that I've redacted the root password, but of course you'll need to type in the actual root password for your appliance):
$ ./sizecalc.py catfish ***** 1
type            NSPF   width  spares   data drives     capacity (TB)
raidz2         False      11       2            22                18
raidz2 wide    False      23       1            23                21
mirror         False       2       2            22                11
stripe         False       0       0            24                24
raidz1         False       4       4            20                15
Note that with only one JBOD no configurations support NSPF (No Single Point of Failure) since that one JBOD is always a single point of failure. If we go up to three JBODs, we'll see that we have a few more options:
$ ./sizecalc.py catfish ***** 3
type            NSPF   width  spares   data drives     capacity (TB)
raidz2         False      13       7            65                55
raidz2          True       6       6            66                44
raidz2 wide    False      34       4            68                64
raidz2 wide     True       6       6            66                44
mirror         False       2       4            68                34
mirror          True       2       4            68                34
stripe         False       0       0            72                72
raidz1         False       4       4            68                51
In this case we have to give up a bunch of capacity in order to attain NSPF. Now let's look at the largest configuration we support today with twelve JBODs:
$ ./sizecalc.py catfish ***** 12
type            NSPF   width  spares   data drives     capacity (TB)
raidz2         False      14       8           280               240
raidz2          True      14       8           280               240
raidz2 wide    False      47       6           282               270
raidz2 wide     True      20       8           280               252
mirror         False       2       4           284               142
mirror          True       2       4           284               142
stripe         False       0       0           288               288
raidz1         False       4       4           284               213
raidz1          True       4       4           284               213

The size calculator also allows you to model a system with Logzilla devices, write-optimized flash devices that form a key part of the Hybrid Storage Pool. After you specify the number of JBODs in the configuration, you can include a list of how many Logzillas are in each JBOD. For example, the following invocation models twelve JBODs with four Logzillas in the first 2 JBODs:

$ ./sizecalc.py catfish ***** 12 4 4
type            NSPF   width  spares   data drives     capacity (TB)
raidz2         False      13       7           273               231
raidz2          True      13       7           273               231
raidz2 wide    False      55       5           275               265
raidz2 wide     True      23       4           276               252
mirror         False       2       4           276               138
mirror          True       2       4           276               138
stripe         False       0       0           280               280
raidz1         False       4       4           276               207
raidz1          True       4       4           276               207

A very common area of confusion has been how to size Sun Storage 7410 systems, and the relationship between the physical storage and the delivered capacity. I hope that this little tool will help to answer those questions. A side benefit should be still more interest in the virtual version of the appliance — a subject I've been meaning to post about so stay tuned.

Update December 14, 2008: A couple of folks requested that the script allow for modeling half-JBOD allocations because the 7410 allows you to split JBODs between heads in a cluster. To accommodate this, I've added a -h option that takes as its parameter the number of half JBODs. For example:

$ ./sizecalc.py -h 12 192.168.18.134 ***** 0
type            NSPF   width  spares   data drives     capacity (TB)
raidz2         False      14       4           140               120
raidz2          True      14       4           140               120
raidz2 wide    False      35       4           140               132
raidz2 wide     True      20       4           140               126
mirror         False       2       4           140                70
mirror          True       2       4           140                70
stripe         False       0       0           144               144
raidz1         False       4       4           140               105
raidz1          True       4       4           140               105

Update February 4, 2009: Ryan Matthews and I collaborated on a new version of the size calculator that now lists the raw space available in TB (decimal as quoted by drive manufacturers for example) as well as the usable space in TiB (binary as reported by many system tools). The latter also takes account of the sliver (1/64th) reserved by ZFS:

$ ./sizecalc.py 192.168.18.134 ***** 12
type          NSPF  width spares  data drives       raw (TB)   usable (TiB)
raidz2       False     14      8          280         240.00         214.87
raidz2        True     14      8          280         240.00         214.87
raidz2 wide  False     47      6          282         270.00         241.73
raidz2 wide   True     20      8          280         252.00         225.61
mirror       False      2      4          284         142.00         127.13
mirror        True      2      4          284         142.00         127.13
stripe       False      0      0          288         288.00         257.84
raidz1       False      4      4          284         213.00         190.70
raidz1        True      4      4          284         213.00         190.70

Update June 17, 2009: Ryan Matthews with help from has again revised the size calculator to model both adding expansion JBODs and to account for the now expandable Sun Storage 7210. Take a look at Ryan's post for usage information. Here's an example of the output:

$ ./sizecalc.py 172.16.131.131 *** 1 h1 add 1 h add 1 
Sun Storage 7000 Size Calculator Version 2009.Q2
type          NSPF  width spares  data drives       raw (TB)   usable (TiB)
mirror       False      2      5           42          21.00          18.80
raidz1       False      4     11           36          27.00          24.17
raidz2       False  10-11      4           43          35.00          31.33
raidz2 wide  False  10-23      3           44          38.00          34.02
stripe       False      0      0           47          47.00          42.08

Jim GrisanzioWebsite Infrastructure: Collective Life Cycle Instructions

June 17, 2009 05:10 PM GMT

I drafted a new document to answer questions about collective life cycle issues on opensolaris.org -- Website Infrastructure: Collective Life Cycle Instructions. The document lives in the Website Community Group and attempts to explain the infrastructure implementation procedures I go through on project-setup for Community Groups, Projects, and User Groups. It also points to all the other documents that are involved in various stages. We can`t unify all those documents at the moment, so I thought that a doc outlining basic procedures and putting the other docs in context would be helpful and would simplify some things. I think I got everything in there for this first cut, but I am sure it will all evolve over time. Glynn Foster drafted an excellent group life cycle document as part of the proposed constitution in March, but that entire effort wasn`t successful so I`ll work it this way and see where we go. At the very least, it helps clarify the implementation part of the process.

Dave MinerUpcoming user group meetings

June 17, 2009 04:58 PM GMT
A couple of user group meetings next week in the northeastern US.  For those in New England, NEOSUG is meeting on June 23.  I won't be there, though, as I'll be in New York for some customer events and will also be speaking about the OpenSolaris deployment technologies at the NYC user group meeting on June 25.  Hope to see you there!

Erwann ChenedeMore information about the File Version explorer

June 17, 2009 10:04 AM GMT

Over at The Observatory, Brian Leonard posted more information on how to use the compare feature I implemented in the file version explorer of Time-slider.

Thanks Brian :) 

In the next version the compare feature will get even more interesting, if time permits ! Watch this space...



Robert MilkowskiVirtualBox 3.0.0 Beta1

June 17, 2009 09:46 AM GMT
This version is a major update. The following major new features were added:

In addition, the following items were fixed and/or added:

Jim GrisanzioDive Deep into OpenSolaris at CommunityOne

June 17, 2009 01:35 AM GMT
Going to CommunityOne in San Francisco next week? Here's the schedule. Don't forget the "deep dives" they have in there, too. You can register for those sessions right here. It's free.