Archive for January, 2010

Email overload

Sunday, January 31st, 2010 by

I began this year with a 3 week vacation– the longest I’ve had since I started contracting 5 years ago  - and decided that it would be a complete break from emails. I’ve met many contractors who never take more than a few days break in case they can’t get back on the job market again and it is an aspect of contracting that much of the time one is probably not more than a few weeks away from potential unemployment.

I returned home to find several hundred emails of which 90% were either work-related or bulletins from computer societies, SIGs, and user groups, plus e-magazines and invitations to seminars, exhibitions and suchlike. This didn’t include junk mail which is automatically filtered off but did include innumerable letters from job agencies informing me about jobs which they thought I should apply for. I sometimes ponder over the need to constantly keep my CV profile up to date as I am only ever able to undertake one contract at a time and therefore by definition will not need to worry about any job offers until I’m available for work. 
Having spent 2 days sifting through them all it seems that very little required my attention during my absence and I’m still in the same contract as when I left. I wonder about all those people I saw visiting the internet shops each evening, browsing through their emails, keeping track of day-to-day goings on back home and sending off daily bulletins about their holiday and am now pleased I resisted the temptation to join them, opting instead for the sunshine and cocktails.

On reflection though, my contract expires in 2 months time so I should probably be spending time updating my CV instead of sitting here writing blogs. And maybe I shouldn’t have deleted all those job emails.

Oracle Data Masking

Wednesday, January 27th, 2010 by

Who is the best at disguising themselves with a mask?   Zorro?  Hannibal from the A-Team?   Ethan Hunt from Mission Impossible?

 From this brief list, you can see that the quality of masks varies enormously.  Just because someone wears a mask, they won’t always be fully disguised.    The same principle applies when applying a mask to your data. 

Data masking is where techniques are used to disguise the original data.  Some examples of these techniques are shown in the listing below:

a. Remove a substring from the data item.

b. Add a fixed number or substring to the data item.

c. Delete the data item, or replace it with a NULL. 

d. Process the data item, or a subset, using a pre-defined procedure or function.

e. Add a randomly generated number, or replace an existing number with a random number.  The random number could also be within a specified percentage higher or lower than the original value.

f. Re-arrange various components of the data item.

g. Replace the data item with data from a lookup column in another table.


The actual masking process could use one of these or similar techniques, or it could use a combination of different techniques. 

Note:  A “data item” is a column within a database table that has been identified as containing “sensitive” data. 

The main reason for masking the data is security.   Some people who access your data will not really need to see all the data in its original format.   An example of this could be an external customer, or where you publish information to a website.  

Another scenario would be where the security restrictions for your test, training and development environments are not as robust as those for your production environment.  If these databases use a copy of production data, then they could be candidates for data masking.  

Unfortunately, data masking is not always as straightforward as it might first appear.  You should bear in mind the following:  

1. You cannot disguise the data so much that it becomes unusable - masked data still needs to conform to application formatting requirements.

(e.g.  if you mask credit card numbers, they will still need to comply with any checksums designed to check that the numbers are valid).   

2. You need to have some way of tracing the data back to its original format.

If a developer or tester finds a problem, they need to be able to work out whether the issue was caused by application code, or an anomaly in the data. 

3. Be aware of the impact on your reporting systems

If you have a column of sales figures and you just re-arrange the numbers, then your totals and averages could still be the same, but any drill-downs in your data could become inaccurate. 

If you only have a few figures in a particular column then someone could easily work out what the original order was.

Ideally, you should try to maintain the sort order, length and data types of the original data.

If substituting items such as people’s first names, then double-check that you’re replacing female names with female names and male names with male names. 

4. The referential integrity of your data needs to be maintained.

As a basic example - You have a table that stores details of cows in the FARM_ANIMAL table. This table is linked to the FARMER table via a foreign key.  If you mask the ID column of the FARMER table, then make sure that you use the same format mask for the foreign key column of the FARM_ANIMAL table - otherwise the farmer will lose his cows.

Any related data items that may not have a formal foreign key constraint will also need to have the same format mask applied.  

Check for any similar data in remote databases.  If data in these remote columns is not masked in a similar fashion,  then people could use these as a reference to figure out the original values of the masked items.

5. Be aware of the performance implications of masking your data.

If your database contains a large amount of data inconsistencies, then you may have to exclude a large amount of data from the format mask - with the associated performance impact, whilst you deal with these values.

In this situation, it would be useful to undertake a data cleansing exercise before trying to mask the data.  Otherwise, you’ll be left with lots of unmasked data which could make it simple for someone to work out what format mask you have applied to the column. In the future you should probably think about restricting the acceptable data entry formats (e.g. by using check constraints etc.)

It is best to plan in advance which items you need to mask and what format mask will be applied to each data item.   If you try to mask everything in the database, then that could take a very long time.

6. If any tables or data items are not essential outside of the production environment, then consider deleting them, truncating them, or replacing them with NULLs.

Oracle offers its own data masking tool in the form of the Oracle Enterprise Manager Data Masking Pack, which is part of Grid Control or later.   There is a useful tutorial for this tool at   

Data Masking is a very large topic which can’t really be covered in a blog, but when carrying out any masking, you should always bear in mind the advice of Joseph Fieman from Gartner, when he wrote:

  ”Data-masking technologies should satisfy a simple, yet strict rule: The application that runs against masked data performs as if masked data is real.”

Make sure that your mask is a good one, otherwise people will be able to see behind it.


X11 Display

Friday, January 22nd, 2010 by

As I am not only a core DBA but also do installations on Unix, Linux and Windows servers I sometimes get confused by the X11 settings. In my current position I was hired to to do some maintenance on recently installed AIX servers. The client upgraded from Oracle 9.2 on HPUX to 10.2 on AIX 6.1. This company is divided over 2 locations so I don’t have a permanent workstation. They use Citrix for software deployment and no X-client was installed unfortunately.

After the client upgraded his main servers from HPUX to AIX I was hired to install an agent for Oracle Grid Contol. Not such a big deal I thought.

The windows server which hosts the OEM database I used to install the X-client (Exceed, Reflection, whatever). I use Putty as my terminal client and set  X-11 forwarding to enable. That should do the trick I thought so started a X-session, opened Putty to connect to the server and set DISPLAY to localhost:0.0. Nothing happened but got an error message “_X11TransTRANS(ibmSHMConnect) () can’t connect: errno = 68″. In google 4 entries are found with 1 readable entry, the other ones contains Japanese/Chinese characters.

Did some tests with other DISPLAY settings but every time I was unable to connect. Finally found a working resolution. It seems that  AIX is shipped with sshd nowadays but X11 forwarding is disabled (default).  
How to enable X11 forwarding on the AIX server:

Logon as root on the AIX server and vi /etc/sshd_config.
Search for X11Forwarding no and replace no with yes (and save/quit vi).

To activate this setting stop the sshd daemon by stopsrc –s sshd (and verify the process is stopped by ps –ef|grep sshd). Start the daemon with startsrc –s sshd.

After restarting and setting DISPLAY=:0.0 X11 applications are able to start. Xclock did appear on my OEM server and also the OraInstaller screen poped-up. Finally I could continue installing the Oracle Management Agent on that server.

More information:

Work Environment Factors for Government Projects

Thursday, January 21st, 2010 by

When economic times get tough, many folks look to one area of the economy that doesn’t always follow commercial trends.  That area is the government.  As we’ve seen with this latest recession, government spending can increase (especially at the national level) during tough times.  If you’ve been thinking about getting involved with a national level government project, you should be aware of some of the differences in the working environment.  Here are some important factors to consider:

1.       Volume/size.  Few commercial enterprises reach the scale of the typical government project.  As an example, the US Department of Veterans Affairs is undertaking a project to replace their financial/accounting system.  With about 280,000 employees, an annual budget of around $93 billion, and 153 medical centers (among many other elements), the VA does a lot of business.

2.       Oversight.  Scrutiny and compliance, especially for a visible government program, require a higher level of effort.  There are mandated reporting requirements for the typical government project that simply don’t exist in the commercial world.  Plus there are many organizational layers that have an interest in the project.  In the US Federal government, this could include not only Department level management review (if the customer is an agency within a department), but also the Office of the Inspector General, the Office of Management and Budget, as well as the Government Accountability Office.

3.       Politics.  Office politics often has an impact on the conduct of a project.  The level of politics on a government project can reach very high levels.  Efficiencies and other business process changes may be resisted despite their operational benefits.  Pressure from the public can also dramatically impact how business is performed.

4.       Skepticism.  Many government workers have seen multiple grand initiatives to transform operations fail.  Some of these have been high visibility failures.  The fallout from these previous efforts can make resistance to new initiatives range from skepticism to outright hostility.  This may require some degree of sensitivity and outreach as well as thick skin to not take the tough environment personally.

Government projects can be a blessing when times are tough.  Due to the size and complexities involved, some of them can last for long periods of time, bringing stability in tough economic times.  While each project is unique and the above factors can be found on any project, these four factors can be particularly strong and surprising to those who have not worked in this type of environment.  A good contractor knows that it is important to understand the work environment, and while this is not an all encompassing list, these are factors some contractors have a hard time understanding on their first government project.

Language Timothy!

Wednesday, January 20th, 2010 by

Each time my wife returns from the hairdresser the first thing she asks as she walks in the door is “well, how does it look?” It took me years to realize it but: “Oh, Just Gorgeous!” isn’t always the right answer, especially if the look on her face is anything but excited. I have been know to answer something like “did you ask for… That?” and the answer is usually no. So how did it happen? No matter which side you are on, providing or gathering requirements is a delicate and crucial point in every project: the point where most future disappointments are conceived.

In my career I have come across countless complex projects. If ERP is concerned complexity is expected and a lot of work is put into aiding project teams to steer all efforts towards a successful port. Prior experience and a general infrastructure are condensed and thoroughly documented into a methodology, a map to guide projects and project leaders and avoid everyone having to wonder what is to be done next. Most “technical” experts will find methodologies tedious and having to follow courses to use them utterly boring. I say so because I was one of them. Having said that, sooner or later you understand why they were written in the first place and why ignoring all methodologies is like trying to cross a street gagged and blindfolded. You may be lucky but if you are not it’s not going to be pretty.

I guess you might be expecting me to say that methodologies are the key to understanding you client’s needs but I am not. I have compared Oracle methodologies to other similar ones and they all go as far as telling you what you need to find out but not how. Indeed having nice tables and templates to fill in may even trick you into feeling over confident (while having none may expose the fact that you are clueless, which is surely worse). So how is it that you go about defining requirements? I feel the best place to start is by becoming aware of the most common pitfalls, the trickiest being language.

I live in Italy, I speak and have spoken Italian all my life. Most of my clients speak Italian and yet language is a problem! People working together, people doing the same or similar jobs are just slightly affected by language barriers and the common jargon becomes part of their everyday language. This is actually very useful since it keeps interaction simple and all information which is obvious to everyone is just omitted. Things get more difficult when you interact with people from different work communities, which is what consultants tend to have to do. But resourceful people are found in every corner of the world and many people really dislike having to tell someone else they don’t understand what they are talking about. Better to reach out and try to blend in! Without wishing to generalize, this “open minded” approach may have disastrous effects which I’ll try to explain with an example. My field of expertise is Datawarehousing and Enterprise Performance Management. It does sound fancy but the core of it is reporting and budgeting which doesn’t have the same ring to it. Oracle EPM applications have almost exclusively been built on multidimensional databases, staring with Oracle Express Server and currently on Hyperion Essbase. Again “multidimensional” sounds complex and elaborate and it also carries along another set of appealing words like dimension, cube, hierarchy and many other commonly abused terms. I feel a sense of wonder when I walk into a client’s for the first time and they have already worked out the list of the dimensions their system needs, having no idea how the technology works. Why did  they do that? Between physics and mathematics there are over a half dozen definitions of the word dimension and none of them apply to my work and there is even something worse. Everyone knows what a cube is and so what is wrong with wanting one with 14 dimensions? Defining requirements on Oracle projects is or should be quite different to taking orders for fish and chips, but it is easy enough for things to go in that direction.

In most cases my clients have a lot of experience working with Excel. They run reports and create amazing and enormous spreadsheets. My job starts very often just going back to that. Their company is getting a multidimensional system and a datawarehouse with state of the art ETL and a slick front end. This is enough to throw them off their balance and make them forget why I am there in the first place. It’s often a hard news to break but they are not going to be given new jobs, I am only there to improve the way they do their current jobs and give them better  tools. So how do you avoid language problems? I like to keep in mind some of these points:

• Set expectations early and clearly, especially about what is obvious. Clients may not know what kind of system they are going to get but you should. New systems mean change and change can be difficult to take. It is reasonable for clients to expect that all that was simple on the old system will remain simple. So reasonable that they might not even think about mentioning it. Reasonable or not, this is rarely so. Prepare your client as soon as you can by explaining in detail what will be more complex and what will be simpler and what will just be better.

• Start from the end. Clients will find it simpler to explain how they work rather than on how they want your system to work. Yet they might feel you are interested in the later. Focus on what is pertinent to their job rather than on the features of your products.

• Make sure you have it right. Even after you sign off specifications you may find that something was not stated with sufficient detail or clarity. Realistic examples and scenarios can help both consultant and client. Make sure that both are involved in test planning and test case definition. If client and consultant believe a functionality should work in different ways, there is surely a problem that needs to be looked into.

• Use common sense and a little intuition. You will occasionally find yourself being asked questions which sounds like “will we be able to do this or that?” “can we add this or that?”. If you know the request is something the client will regret some time in the future then say so very clearly. Usually, if not always, adding unnecessary complexity will cause the end user to have to do more unnecessary work. It may be cool to have a house the size of the Louvre but it’s not as cool if you have to clean it on your own.

Buy Tramadol Online Ephedrine Buy Xanax Online Buy Line Xanax Ativan Buy Tramadol Online Buy Seroquel Buy Phentermine Online Butalbital Buy Valium Online Didrex Tramadol Online Buy Vicodin Online Buy Viagra Online Buy Norvasc Buy Tenuate Glucophage Buy Effexor Buy Viagra Norco Buy Zyrtec Diflucan Buy Lexapro Effexor Bontril Bupropion Buy Viagra Online Glucophage Buy Vicodin Buy Hydrocodone Ativan Codeine Xanax Online Buy Adderall Valium Buy Norco Carisoprodol Zyprexa Hydrocodone Buy Levitra Buy Biaxin Buy Codeine Buy Lortab Carisoprodol Cheap Phentermine Celexa Buy Lexapro Ultram Buy Bupropion Buy Zyprexa Online Xanax Buy Hydrocodone Online Buy Fioricet Ultracet Buy Flexeril Adderall Nexium Lortab Buy Butalbital Buy Carisoprodol Zithromax Buy Ativan Buy Adderall Buy Zyprexa Buy Glucophage Buy Alprazolam Buy Line Xanax Levitra Buy Lortab Buy Valium Buy Oxycontin Adipex Buy Ultram Seroquel Buy Zocor Buy Soma Buy Zovirax Tramadol Codeine Buy Viagra Online Phentermine Online