Good way to use the concurrent data store

Hello

I'm developing a multithreaded C++ application that uses the C++ of Berkeley DB Library.

In my case, I have several databases that I composed in a wooded area. It is important for me to use an environment because I need control over the cachesize parameter.

I don't need no guarantee of transaction and have for most of the readings, so I decided to use the "simultaneous database.

I first pre-fill all databases with a number of entries (configuration single-threaded phase) and then work on it at the same time (for most readings, but also insertions and deletions).

I tried all kinds of different configurations, but I can't work without specifying DB_THREAD as a flag of the environment.

I don't want to because then access all handles is synchronized sequentially according to the documentation:

"... Note that the activation of this indicator will serialize calls to DB using the handle between the threads. If

simultaneous scaling is important for your application, we recommend handles separate for each thread opening

(and do not specify this indicator), rather than share handles between threads. "

(Berkeley DB QAnywhere C++)

So I tried to open the environment with the following indicators:

DB_CREATE | DB_PRIVATE | DB_INIT_MPOOL | DB_INIT_CDB

All data in this environment handles are open only with the DB_CREATE flag.

So, since my understanding this same basic access handles need to be synchronized, I opened separate handles foreach database for each thread (opening the handles is still single-threaded).

In my first approach, I have only made use of the object of global environment. Which does not work and gives the following during operations error message:

DB_LOCK-> lock_put: Lock is no longer valid

So I thought, since the same handle global env is passed to all handles separate DB, it is perhaps a race condition critical on the handful of approx.

So in my next test, I opened also handles separate EPS in each thread (and db handles owned each).

That does not produce an error in db, but now it seems that each thread sees its own version of the databases (I call it stat early in the life of each thread and he sees all of the empty dbs).

What is the right way to use the concurrent data store? Each thread should really open his own set of db handles? What about the number of open handles env?

PS: Without specifying that the DB_PRIVATE flag seems to do the job, but for performance reasons, I want all operations to perform in the cache and do not specify product DB_PRIVATE average of several writes to the disk for my scenario.

Thanks a lot for your help.

CD (simultaneous database) allows a single editor with multiple drives, access to the db at a given point in time.    The handle for the writer doesn't have to be shared with readers.   If you share the DB handle then calls are synchronized, but if each thread has its own handle DB then this is not the case.     Since you have an environment, DB_THREAD must be at the level of the environment.   This will allow the sharing of the environment handle.     This type of error "DB_LOCK-> lock_put: Lock is no longer valid" you can provide us your code so we can take a look.   Also what BDB version are you using?

Tags: Database

Similar Questions

  • What is a good way to use the queues for the model of consumers/producers?

    Hi all

    I am following the model of consumers/producers to use the queue to synchronize the following process: the producer is a loop to produce a number N, I will put each number generated in a table and after each 5 numbers generated, I put the table in the queue and pass it on to the consumer. I have to wait the use by consumers of the data and it will then remove the item from queue so that producers will have the chance to produce another 5 numbers. As I put the maximum size of the queue one, I expect that the producer and the consumer turns to produce / consume all five numbers and the opportunity to another. Here is my code

    When the checkbox is false, the code will be

    For the first 5 numbers, product will generate every thing right and put it in the table, and it's going to pass the array to the quere so that the consumer will have the chance to loop through the table. I except the procude loop will continue only when the queue is available (i.e. all items are deleted), but it seems that once the consumer starts the loop loop of the product will continue (if the indicator x + 1 and x 2 will be changed to numbers). But this isn't what I want, I know there must be something wrong, but I can't say it is.

    dragondriver wrote:

    As you say in 1, sequency structure to enforce the order of execution, that's why I put it here, in this example, the simple question, I replace the complete code with increase in the number, in the real case, the first markers + 1 and + 2 must be performed in this order.

    Mikeporter says:
    1. get rid of all the structures of the sequence. None of them are nothing but apply a work order which would have been the same without them.

    So even if you delete the sequence structure, there will be a fixed & defined order and it is because LabVIEW follows the MODEL of FLOW OF DATA.

    Data flow model (more precisely in the context of LabVIEW): a block diagram node runs when it receives the required inputs. When a node is running, it produces output data and transmits data to the next node in the path of the data stream. The flow of data on the nodes determines the order of execution of the VIs and functions on the block diagram (click here for reference).

    Now in your code, just remove the sequence structure will not make you order will be going to stay the same, but you need to do some very minor changes (as thread of the error in loop, before that he go to the node "Élément Dequeue").

    Come to the main point: it's a good way to use the queue for the consumer/pmodel that?
    The model you are using (and qualifying as consumer/pmodel) is much too deviated from the original consumer/pmodel which model.

    dragondriver wrote:

    For the second, Yes, it's my fault for delete, though. I'm actually the example of model of producer/consumer design pattern, but I do not pay attention to the while loop in the part of the consumer.

    While loops (two producers & consumers) are the essential part of this architecture and cannot be deleted. You can start your code using standard model.

  • Deployment uses the local data store

    The deployment of VIO works without problem, but one of the virtual machines (VIO-DB-0) is now in a local data store. I never chose this data store in the installation process.

    Is it possible to manually move the virtual machine? Is it possible to define the data store used for the management of virtual machines somewhere?

    Concerning

    Daniel

    Hi Daniel,.

    Take a look at VMware integrated OpenStack 1.0 Release Notes:

    • Installer gives priority to local storage by default
      When you set up the data for the database stores three virtual machines, VMware OpenStack Setup integrated automatically gives priority to a local storage to improve IO performance. For resilience, users might prefer a shared storage, but the installer does not clear how to change this setting.
    • Workaround: Before completing the installation process, the installer of VMware OpenStack integrated allows you to examine and change the configuration. You can use this opportunity to change the configuration of the data to the database store three virtual machines.

    If you have already installed VIO, AFAIR you should be able to manually move between data warehouses as long as you do not change VM itself.

    Note that there is rule anti-affinite, so you may not be able to move that VM in the same data store when no other VIO - DB resides.

    Another note is that if you turn off VIO-DB-0, mysql service will not appear automatically. you have to turn on manually by running "service mysql start" on the node itself, or run 'vioconfig start' of the SGD server.

    Best regards

    Karol

  • Concurrent data store uses synchronous writes?

    When you specify the DB_INIT_CDB flag in opening a dbenv (to use a concurrent data store), you cannot specify any other indicator except DB_INIT_MPOOL. Does that mean that logging and transactions are not enabled, and in turn this db does not write to disk synchronous? It would be great if someone could confirm...

    Hello

    Indeed, when configuring CDS (concurrent database) the only other subsystem that you can initialize are the buffer (DB_INIT_MPOOL) shared memory pool subsystem. Applications where there is no need of recovery complete or transaction semantics, and where you need help for free from deadlock, multiple-reader/single writer access to the database of CD suites.
    You will not initialize the transaction subsystem (DB_INIT_TXN) or recording subsystem (DB_INIT_LOG). Note that you cannot specify configuration recovery indicators at the opening of the environment with DB_INIT_CDB (DB_INIT_RECOVER or DB_RECOVER_FATAL).

    I guess that by synchronous/asynchronous writes, you are referring to the ability to use DB_TXN_NOSYNC and DB_TXN_WRITE_NOSYNC for transactions in TDS applications to influence the default behavior (DB_TXN_SYNC) when the validation of a transaction (i.e. synchronously clear the log when the transaction is committed). Since there is no stamp journal in a CD set up, no newspapers or operations these flags do not apply.
    The only aspect related to the Scriptures in the CDS applications that needs discussion is the cache dump. Flushing (or the database cache) - DB-> sync(), DB-> close(), DB_ENV-> memp_sync() - ensures that any changes made to the database are wrote the disk (stable storage). Thus, one could say that since there is no guarantee of sustainability, including the recovery after failure, disk writes in demand for CDS are not in sync (they do not reach the stable storage, you must explicitly clear the cache of the environment/database).

    More information on applications of CDS is here:
    [http://www.oracle.com/technology/documentation/berkeley-db/db/programmer_reference/cam.html]

    Kind regards
    Andrei

  • Clone & new Config on the different data store

    Hi all

    I had a problem where the user try to clone or create a new configuration where the original VM model are stored is full.

    When the data store is almost full, I turn off the "VM creation" in the section resources, I would do this or the LM will be able to use the different data store? otherwise how can I config for the user can store data different from Julie then create a new configuration or a clone.

    Thank you

    Error message here: -.

    * "Clone VM" operation cannot be performed because the data store 'VMFS02' is not

    Enabled.* yes I disable 'VM Creation' resources because of this LUN is almost exhausted.

    In this case, you must use SSmove to migrate the entire tree of virtual machines from a store of data with more free space

  • remove the Cluster data store data store

    I have an infrastructure with vCenter and ESXi 4 5.5 I have a data cluster store in SAN with 8 Mon, I need to remove 3 Lun (to be used for other purposes) what is the appropriate procedure

    to remove the Lun (end then to destroy)? Thank you

    Do you want the LUN to use for purpose of non-vSphere? If so, you can just storage vMotion virtual machines since associated LUN data warehouses that you want to decommission (or simply putting the data store in maintenance mode, in this way, that the virtual machines will automatically be migrated). Cleaning after the data store, move the data store from the cluster data store, and then delete the VMware environment data store as described here: best practices: how to properly remove a unit number logic of a host ESX - VMware vSphere Blog - VMware Blogs

  • Disk groups are not visible cluster. vSAN datastore exists. 2 guests (on 8) cluster do not see the vSAN data store. Their storage is not recognized.

    http://i.imgur.com/pqAXtFl.PNG

    http://i.imgur.com/BnztaDD.PNG

    Do not know how even tear it down and rebuild it if the disk groups are not visible. The discs are in good health on each host storage adapters.

    Currently the latest version of vCenter 5.5. Hosts running 5.5 build 2068190

    Just built. Happy to demolish and rebuild. Just do not know why it is not visible on the two hosts and the disk groups are only recognized 3 guests when more are contributing. Also strange that I can't get the disk groups to fill in vCenter. I tried two different browsers (chrome and IE).

    I have now works.

    All the identical 5.5 relies on ESXi hosts. All hosts are homogeneous CPU/sum of the prospects for disk controller / installed RAM/storage.

    I have work. I had to manually destroy all traces of the vSAN on each single to help host node:

    (1) put the hosts in maintenance mode and remove the cluster. I was unable to disable vSAN in the cluster, I made on each node host (manually via the CLI below) then disconnected web client vCenter and return to finally refresh the ability to disable on the cluster.

    esxcli vsan cluster get - to check the status of each host.

    esxcli vsan cluster drop - the vSAN cluster host.

    storage of vsan esxcli list - view records in the individual host group

    esxcli vsan storage remove-d naa.id_of_magnetic_disks_here - to remove each of the disks in the disk group (you can ignore this using the following command to remove the SSD only falling each disc in this host group).

    esxcli vsan storage remove s naa.id_of_solid_state_disks_here - this is the SSD and all the magnetic disks in a given disk group.

    After that, I was able to manually add hosts to the cluster, leave maintenance mode and configure the disk groups. Aggregated data of the vSAN data store is correct now, and everything is functional.

    Another question for those of you who still read... How to configure such as the VM storage strategy that migrates towards (or inspired) the vSAN data store will immediately resume the default storage policy, I built for VSANs?

    Thanks for anyone who has followed.

  • ESXi 5.0 servers don't recognize that same NFS share to the same data store

    I have a setup of vSphere 5.0 running VMware Workstation 8.  I have two ESXi servers that are accumulating the same NFS share (with the same IP address and the same directory path), but for some reason, they don't recognize it as the same data store.  I deleted and added to the data store again and checked that I am getting the same way on both nodes.  However, vSphere don't let me use the same name when I ride sharing NFS as a data store on the second node. vSphere changes the name of "NFSdatastore" to "NFSdatastore (1)" and will not let me change, say that the name already exists.

    The NFS file server is a VM of 2.99 OpenFiler, which IP 192.168.0.100 (and is in DNS as "openfiler1").  The path is "/ mnt/vsphere/vcs/datastore.

    The NFS mount works great as a data store on the two servers ESXi, except for the fact that vSphere sees two data stores.  I have a Red Hat VM running on each of them, and when I browse the data of each ESXi Server store, I can definitely see two configs VM, as well as a few ISO images that I put on the NFS share.  However, I can't vMotion as he complains about not being able to access the configuration file and the file virtual disk (vmdk) of the other node.

    On the OpenFiler, the directory/mnt/vsphere/vcs/datastore is the property of "ofguest" in the "ofguest" group, and the permissions are "drwxrwsrwx +".  All files under the directory are also held by "ofguest" in the group "ofguest".  Vmx files have permissions "rwxr-xr-x + ', and the vmdk files have permissions ' rw +-

    I did a lot of Google search, read best practices, etc.  Any help is appreciated.

    UPDATE: he Figured out - leakage / on one of the paths of NFS mounting.  DOH!

    Maybe this can help you:

    http://KB.VMware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalID=1005930

    http://KB.VMware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalID=1005057

    Concerning

  • Can I delete the local data store on esx 3.5?

    I have a client with vmware ESX 3.5 no esxi

    They have a local data store but it is formatted with a block size of 1 MB, so I can put only a 256 GB VM out there and I need to put a test VM with a larger size.

    I know that in ESXi 4 you can simply delete the local data store and create a new, but I'm not familiar with ESX 3.5, are there hidden files on the data store that the server uses?  If I search for the store there's nothing in there.

    Thank you

    Mike

    Hi Mike,.

    You can simply reformat your data store. Take a look at the following article:

    Of course, you must back up and delete any data you have on this data store before the format operation.

    Good luck!

    Concerning

    Franck

  • Cannot delete the local data store empty on ESXi 4

    Hi all

    Does anyone know how to kill the process that uses / block the process of removing the ESXi 4 data store?

    I have this provblem in VI Client when I was in the middle of the removal of the Local data store

    Remove datastore

    The resources ESXi02_VMFS vim.host.DatastoreSystem is in use.

    It's production live environment so is possible to kill the process without having to restart the ESXi 4 crowd?

    Thank you.

    Kind regards

    AWT

    OK so first try to connect with viclient to ESXi directly and see if you can remove the data store.

    The command you need to nod your head do buisness hours.

    Good luck!

    MCP, VCP

  • Move-VM error: VM must be managed by the same VI servers as the destination data store

    I work with 2 ESXi servers and a single vCenter server with vSphere 6.0 Update 1 b; and PowerCLI 6.0 Release 2

    I have an a datacenter, cluster, vCenter and a vDS.  I need to pass the vCenter data store to another and get the error message:

    "Move-VM VM you are moving should be handled by the same server of VI as the destination container and the data store.

    Code looks like the following:

    $vctObj = to connect-viserver $vctIPaddress

    $esx1Obj = to connect-viserver $esx1IPaddress

    $esx2Obj = to connect-viserver $esx2IPaddress

    $vmObj = get-vm-name $vmname - Server $vctObj

    $destDSobj = get-datastore-name $destDSname

    Move-vm - vm $vmObj Datastore - $destDSobj - 'End' DiskStorageFormat

    I can migrate successfully from the virtual machine to vCenter data store through the VI Client, but PowerCLI left me speechless.

    Maureen

    Try to use the Server parameter on the Get-Datastore line

  • Extremely high latency during the migration of the local data store to data store shared.

    Hi guys I hope you help me. Sorry for my English btw, I'm not native.

    Let's start!

    I have:

    1 vCenter
    1 host
    1 Distributed Switch (with a pg for managing network/IPstorage the esxi)
    1 standard switch (empty)
    1 FreeNAS to provide iSCSI LUNS
    1 Microsoft to provide iSCSI LUNS

    When I try to migrate virtual machines between the warehouses of shared data or data shared at local store, all right. The problem has become when I try to migrate virtual machines from the local to the shared data store. All data stores down (all paths down) and back up and I get this error:

    "Error caused by the /vmfs/volumes/volumenID/VMDirectory/Disk.vmdk file.


    When I try to migrate virtual machines from the local to the FreeNAS iSCSI data store it fails immediately.
    When I do the same local Microsoft iSCSI datastore that it takes a time loooooong to migrate the virtual computer, give it to me even all the paths downwards and uplinks down error but not lacking in migration.

    I'll give you some screenshots to see the errors.

    Thank you very much!

    EDIT: I have extremely high latency notice when I try to migrate from the premises of warehouses of shared data. average 2000ms with peaks of 50,000 (see my response below for more information)

    Finally, I found the solution! The problem is that I have been using E1000E vmnic instead of vmxnet3. I have configured an adapter vmxnet3 and boom! 20 ms in all migration!

    Thank you to all for help, especially Nick_Andreev !

  • VM shows this in the NFS data store after passed to the vmfs data store

    Hello

    (1) create a virtual machine using NFS datastore

    (2) had created a few snapshots

    (3) to migrate this virtual machine from the data to the local VMfs store

    (4) all the files are moved to the defined vmfs datastore

    (5) storage > > nfs_datastore > > objects > > VM > > vm is present

    (6) but when did change the setting of this VM, vm is present on the data store local vmfs

    The .vmsn file contains binary data text, so you better not change it manually. The purpose of these files is to be able to return to his instant partner. Depending on the configuration of virtual machines in time you created the snapshot, the virtual machine may require access to the old data store to access the files and folders in there when you return to this snapshot.

    André

  • How to migrate Office persistent link clones of workstations for the new data store

    Hello. We have clones of persistent link to view running in the environment from our view. We need to migrate these desktops to new storage. Rebalancing is out as this will include a refresh operation - destroy anything and everything installed in c:\ of the user. Is there a method to migrate Desktop clone link persistent to a new data store, while keeping the office in the same exact state (i.e. c:\ is not updated).

    Thank you

    Never say impossible. I came across the solution when I read the article by Duncan Epping:

    http://www.yellow-bricks.com/2012/03/15/Dr-of-view-persistent-linked-clone-desktops/

    I have that for my situation. Basically, replicate data from view in the new storage warehouses. Set up data warehouses replicated without re-signing with the cluster of ESXi. Turn the power off and unsubscribe the vCenter view workstations. Save the workstations from the data store replicated/mounted to vCenter and power over desktop computers.

    VCenter, nothing dramatic has happened if she sees just as vm being unregistered and re-recorded activity from the same exact data store (the same UUID). View Admin is happy because he sees just the events as desktop computers that have been turned off and then subsequently in reverse. Nothing has changed regarding the database to view ADAM, we still use the same data warehouses.

    I have already tested before for a scenario of View DR, and he succeeded. Between remembering who and Duncan's article helped me to adapt the same process for this. Nothing is impossible

  • Thickness of thin formatting on the same data store?

    Hi all

    is it possible with vCenter to convert thick to thin disc on the same data store format?

    and how?

    Thank you very much

    VMware KB: Change the thick or thin provisioning of a virtual disk

    Please check the useful and accurate.

    Marc

Maybe you are looking for

  • Firefox 4

    I was looking for Firefox 4 because I want to do some browser testing. I couldn't find it anywhere, even on the old browser page supposed to: http://www.Mozilla.com/en-us/Firefox/all-older.html

  • playlist does not work all of a sudden without known reason?

    I placed my playlist with my songs together in a new folder in the folder "music". In the beginning, it has worked. Then I created another folder in the 'Music' folder and Yes, the new songs in the folder and another reading list. After you have disc

  • I have a problem with the password is not eligible not on computer.

    Original title: password problem I have a toshiba laptop and when I put in place I used the same username and password on all so one day I went to get my desktop to defragment my computor and my password does not work but it always works for everythi

  • Compatible Snaptics Ps/2 Port touchpad stop working, gives error Code 24

    Last Thursday my Snaptics Ps/2 Port Compatible TouchPad stop working. Code 24 arrives. What is the 24 code and how can I solve this problem?

  • BlackBerry smartphones to remove several Messages?

    First of all, I have the storm. It is not perfect,... and has a way to go, but it's a whole new product and design for Blackberry. Be a software QA tester, I see it all the time. Give them people of the time, I do not know marketing that knocked out