Managing Data

May 19, 2009

Using libmysqld with Microsoft Visual C++ 2008 Express

Filed under: Uncategorized — lstigile @ 7:46 pm
Tags: ,

[ Note: this post was updated on Dec 2, 2009 to correct dashes. In the original post, dash dash -- was showing up as a single dash. This is noted in the comments and has been corrected in the text below.]

These are very (very) basic instructions for setting up a very simple application with Microsoft Visual C++ 2008 Express to use the MySQL Embedded Library (libmysqld).

The MySQL Embedded Library is a full version of the server that is available as a dynamic or static library.  Developers can run the library inside their application.  It’s extremely fast, easy to distribute and it’s ideal for stand-alone applications.  More information about the library can be found at http://dev.mysql.com/doc/refman/5.1/en/libmysqld.html

Having little experience using Visual C++, I’ve struggled to find instructions for setting up a project to use a dynamic library.  I finally got it to work thanks to Ulf Wendel’s blog about building a windows client using the Connector/C++ ( http://blog.ulf-wendel.de/?p=215 ).  I was able to take his tutorial, substitute the appropriate libmysqld entries and compile the project.  If you get lost following my instructions, refer back to Ulf’s blog.

To develop a libmysqld application, you’ll need data files, the error message file ( errmsg.sys ), the appropriate libraries and header files.  To get these files, I’d recommend downloading and installing the full version of the server.  Also, it’s easier to design and build your database using the standard version of the server.  Once you have some data, stop the MySQL service and then create your application using libmysqld.

Here’s the process that I used.

Download the Windows MSI Installer (x86) from http://dev.mysql.com/downloads.    For this example, I downloaded and installed version 5.1.34.  I selected the option for a standard install.  After the install, you can create tables and design your database as needed.  For this example, I’m going to use the “user” table, which installs with MySQL.

After the install, go to the services control panel ( Start | Control Panel | Administrative Tools | Services ) and stop MySQL.  You might also change the Startup type to “manual” to ensure that it doesn’t start on reboot and cause a conflict with your project.

Locate required directories for setting up the project:

1)  Data Files

On Windows, the default location for data files is C:\Documents and Settings\All Users\Application Data\MySQL\MySQL Server 5.1\data  We’ll be accessing the user.MYD MyISAM table, which is located in the .\mysql subdirectory.  Similarly, If you were to create a database named world with a city table, there would be a corresponding world directory and city.MYD MyISAM table in the data directory.

2) Error Message File

The error message file ( errmsg.sys ) is required to start libmysqld.  By default, it’s located at C:\Program Files\MySQL\MySQL Server 5.1\share\english

3)  Header Files

C:\Program Files\MySQL\MySQL Server 5.1\include

4) Embedded Library

C:\Program Files\MySQL\MySQL Server 5.1\Embedded\DLL\release

Copy the libmysqld.dll from the embedded library path (above) to the c:\windows\system32 directory (or anywhere in your search path).  For deployment, you’ll probably want to include the embedded library with the executable.

Time to setup up Microsoft Visual C++ Express

Launch Microsoft Visual C++ Express.  From the main menu, select File | New | Project

Choose a Win32 Console Application

g1

Select OK and Finish.

Right-click the project name in the Solution Explorer and select Properties.

g2

Expand Configuration Properties, C/C++ Properties and add the following for the Additional Include Directories:

C:\Program Files\MySQL\MySQL Server 5.1\include

g3

Expand Linker, select General and add the following to Additional Library Directories: C:\Program Files\MySQL\ MySQL Server 5.1\ Embedded\ DLL\releaseg4

Select Linker, Input and add the following for Additional Dependencies: libmysqld.libg5

Select OK to save the setting and return to the project.

Copy the following into the source file ( my copy is named sample.cpp )  Overwrite any existing code in the file.

#include “stdafx.h”

#include <stdlib.h>

#include <iostream>

#include <stdio.h>

#include <stdarg.h>

#include <windows.h>

#include <mysql.h>

MYSQL *mysql;

MYSQL_RES *results;

MYSQL_ROW record;

static char *server_options[] = { “mysql_test”, --datadir=C:/Documents and Settings/All Users/Application Data/MySQL/MySQL Server 5.1/data,

--language=C:/Program Files/MySQL/MySQL Server 5.1/share/english",  NULL };

int num_elements = sizeof(server_options)/ sizeof(char *) – 1;

static char *server_groups[] = { “libmysqld_server”, NULL };

int main(int argc, char* argv[])

{

int retval;

retval = mysql_library_init(num_elements, server_options, (char **)             server_groups);

mysql = mysql_init(NULL);

mysql_options(mysql, MYSQL_OPT_USE_EMBEDDED_CONNECTION, NULL);

mysql_real_connect(mysql, NULL,“root”,NULL, “mysql”, 0, NULL, 0);

mysql_query(mysql, “SELECT user, host FROM user”);

results = mysql_store_result(mysql);

while((record = mysql_fetch_row(results))) {

printf(“%s – %s \n”, record[0], record[1]);

}

mysql_free_result(results);

mysql_close(mysql);

return 0;

}

At this point, you should be able to build and run your project.  The output should look like this.

g6

It’s a simple listing of the users in the mysql table.

The following line of code is critical:

static char *server_options[] = { “mysql_test”, --datadir=C:/Documents and Settings/All Users/Application Data/MySQL/MySQL Server 5.1/data,

--language=C:/Program Files/MySQL/MySQL Server 5.1/share/english",  NULL };

You must have a datadir and a language file.  Note the forward slash in the directory paths.  Of course, these paths can be located anywhere.  You’ll probably want to move these folders to a subdirectory under your application install.  If you don’t see an error log in the data directory, that’s also an indicator that these paths are incorrect.

I hope someone finds this useful.

June 30, 2008

The Case for MySQL Cluster Certification

Filed under: Uncategorized — lstigile @ 9:46 pm
Tags: , , ,

Why are the best technical books also the thinnest? To make the case, I highly recommend the MySQL 5.1 Cluster Certification Study Guide In rack units, it’s a 1U; it fits nicely into my laptop bag; and if you’re considering implementing MySQL Cluster, it can save you a world of time.

Of course, if you get the book, you should consider the certification itself. There’s a legitimate debate about the usefulness of certification exams, but the MySQL 5.1 Cluster certification is a little more important than others for a couple of reasons:

1. In case you haven’t heard it by now, Cluster isn’t always the best fit. In many cases, an active/passive failover setup is a much more flexible and cost-effective approach. It’s not always a clear-cut decision. If you’re willing to make application changes and if you understand the internal workings of MySQL Cluster, it can be an amazing solution.

To see if cluster is a good fit, I recommend the following guides:

http://www.mysql.com/why-mysql/white-papers/mysql_cluster_eval_guide.php

http://www.mysql.com/why-mysql/white-papers/mysql_ha_solutions.php

2. There aren’t many consultants and DBA’s that have actually installed and used MySQL Cluster. There’s still a lot of misconceptions. The primary misconception is: “I know MySQL, I’m sure that I can quickly set it up and support a MySQL Cluster.” Outside of MySQL professional services group, there’s limited expertise in the marketplace. If someone has a MySQL Certification, at least you know that they’re tall enough for this ride.

I thought the most important part of the book was Chapter 7 regarding Indexes in NDB tables. Unless specifically suppressed, a hash and ordered (T-tree) index will be created for primary keys. The hash index is stored in the IndexMemory pool. The ordered index is stored in the DataMemory pool.

If you want to create a unique index, that’s a bit more complicated and it takes about a page to explain. Unique indexes have performance implications, which provides even more reason that someone should really understand MySQL internals before implementing a cluster.

To summarize, I’m hesitant to say this in public, but frankly; the study guide is a good read. It won’t make the Oprah book club any time soon, but it has some good discussions about hash indexing vs. t-tree indexes, database isolation levels, ACID properties and high availability concepts.

[sidenote: I usually rip technical books in half to achieve 1U. Usually, you can safely throw the first half away.]

December 27, 2007

Tune Drupal, XOOPS, Joomla!, Alfresco…with MySQL Enterprise Monitor

Filed under: Uncategorized — lstigile @ 10:10 pm

Have you ever tuned the MySQL instance underlying your content management system (CMS)? For most users of Drupal, Joomla!, XOOPS, Alfresco, or any of the other CMS, the answer is no.

However, I’m seeing an increasing number of mission-critical corporate sites running on open source CMS. Take a look at Alfresco’s customer listing, http://www.alfresco.com/customers , which includes H&R Block, Boise Cascade and Harvard Business School Publishing. If your CMS site has become a crucial part of your enterprise, it’s time to take a closer look at the database.

The MySQL Enterprise Monitor is designed to help developers configure, tune and administer their MySQL Servers. After watching a Drupal presentation in a local meetup, I decided to evaluate the Monitor with Drupal.

Test Scenario
1. Install Drupal and MySQL
2. Install the MySQL Enterprise Monitor ( A trial can be downloaded from http://www.mysql.com/ent-trial-reg-2007 )
3. Add a load to the site using DieselTest ( http://www.dieseltest.com )

MySQL Enterprise Monitor
The Monitor has two components: the agent, which collects statistics (600+) from the production server; and the dashboard, a web-based interface, which is installed on a non-production machine within the firewall. The dashboard can be used to determine what gets monitored (70+ options), to configure email notifications and to view 20+ graphs.

Figure 1.Figure 1.

Heat Chart
By default, the Monitor includes 12 rules that comprise the heat chart (figure 1). The heat chart is on the front page of the web-interface and provides a one-glance status report on your MySQL server. It monitors if the server is up or down; the I/O, CPU and RAM usage; database connections; key and query cache; table scans; temporary tables-to-disk ratio and lock contention.

Critical Alert! -Query Cache Has a Sub-Optimal Hit Rate
The Monitor instantly threw a critical alert that the Query Cache Has a Sub-Optimal Hit Rate. The query cache holds not only the SQL statement, but the data as well. Results for identical statements can be instantly retrieved from cache. However, it also means than any updates to the underlying table will force the cache to be invalidated. Since my load test was primarily duplicate reads, I should probably follow the advice of the alert and increase query_cache_size.

Security and Administration
With version 1.2, there are 18 security rules and 16 Administration rules. These rules will provide email notification for issues ranging from the existence of anonymous user accounts to an excessive number of attempted connections (DOS attack?). I actually received 3 administrative alerts and 7 security alerts. I had covered the basics, but it still triggered several warnings including advice to disable symlinks and to disable older (weaker) 4.1 password authentications.

Adding a Load to Drupal
I browsed the internet and selected Dieseltest (http://www.dieseltest.com) for load generation. Dieseltest looked easy, and it was. I simply recorded some blog and forum updates and some browsing on my drupal site. Then, I re-played the script with 50 virtual users for 10 minutes. Of course, this was a fairly rudimentary process, but it did add load to my database server (see figure 2).

Figure 2. Figure 2.

Performance
In all, I received nine performance events. I outlined the two most critical below:

Critical Alert! -Indexes Not Being Used Efficiently
As I suspected, the default database could use some indexing. See figure 3 for rows read via indexes versus rows read with a full table scan. As the alert suggested, I need to enable –log-queries-not-using-indexes.

Figure 3.Figure 3.

Critical Alert! – Excessive Disk Temporary Table Usage Detected
I may need to do more research on this one. I suspect that the temp tables are a result of using a lot of TEXT data types for storing blog entries. Perhaps I can change some data types to varchar so that disk temporary tables aren’t automatically used. The alert provides plenty of detailed information for adjusting parameters and links to more online references.

Conclusion
A web site developer recently told me that he designed his sites without ever changing the default MySQL installation. While this speaks volumes to the ease-of-use of MySQL, if your site becomes mission-critical or has grown to handle more traffic than you anticipated, consider installing the MySQL Enterprise Monitor for configuring, monitoring and tuning your server.

« Previous Page

Theme: Rubric. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.