96

When you use a map in a program with concurrent access, is there any need to use a mutex in functions to read values?

2
  • 4
    If it is strictly a read-only map, a mutex should not be necessary. Commented Jun 16, 2012 at 12:40
  • I was not very clear since there will be functions to set and get values. Commented Jun 16, 2012 at 12:47

6 Answers 6

124

Multiple readers, no writers is okay:

https://groups.google.com/d/msg/golang-nuts/HpLWnGTp-n8/hyUYmnWJqiQJ

One writer, no readers is okay. (Maps wouldn't be much good otherwise.)

Otherwise, if there is at least one writer and at least one more either writer or reader, then all readers and writers must use synchronization to access the map. A mutex works fine for this.

Sign up to request clarification or add additional context in comments.

2 Comments

What about multiple writers, no readers? Would that result in memory corruption?
@user3125693 Sure.
70

sync.Map has merged to Go master as of April 27, 2017.

This is the concurrent Map we have all been waiting for.

https://github.com/golang/go/blob/master/src/sync/map.go

https://godoc.org/sync#Map

4 Comments

Nice one. Do note that the new sync.Map type is designed for append-only maps (for this reason it doesn't use sharding of keys, which means that if your map has a lot of churn it will most likely underperform sharded style maps as per my answer above).
@OmarIlias I mean cases where you mainly set new values (without editing or deleting existing values). See this: github.com/golang/go/issues/20360
@orcaman current go sync.Map can be faster or slower than hash-sharded even in your append-only case. It really depends on the scenario. Using as much atomic operations as possible, the sync.Map internals can be much faster than traditional sharding since it minimize locking versus traditional hash-bucket but required locking. I am sure there will be benchmarks performed which detail the sweet and sore spots of the new sync.Map implementation. But to assume is slower is not correct especially factoring concurrency.
@Diegomontoya of course, here it is: medium.com/@deckarep/…, in brief sync.map will be faster only if the number of cores used is bigger than 4, if not the case, using only mutext will be 4 times faster at maximum
25

I answered your question in this reddit thread few days ago:

In Go, maps are not thread-safe. Also, data requires locking even for reading if, for example, there could be another goroutine that is writing the same data (concurrently, that is).

Judging by your clarification in the comments, that there are going to be setter functions too, the answer to your question is yes, you will have to protect your reads with a mutex; you can use a RWMutex. For an example you can look at the source of the implementation of a table data structure (uses a map behind the scenes) which I wrote (actually the one linked in the reddit thread).

7 Comments

it's usually wasteful to use a full reader-writer lock for a resource as fast to access as a map
Can you elaborate a bit please? What would be better suited?
RW locks are good for resources with a lot of contention, but they have more overhead than a mutex. Map get/sets are fast enough that the program likely won't have enough contention to make the more complex lock give better throughput than a simple mutex.
Thank you for the clarification. Do you have any papers/articles that you can recommend on this matter?
However, the Concurrency section of this post blog.golang.org/go-maps-in-action suggests sync.RWMutex
|
23

You could use concurrent-map to handle the concurrency pains for you.

// Create a new map.
map := cmap.NewConcurrentMap()

// Add item to map, adds "bar" under key "foo"
map.Add("foo", "bar")

// Retrieve item from map.
tmp, ok := map.Get("foo")

// Checks if item exists
if ok == true {
    // Map stores items as interface{}, hence we'll have to cast.
    bar := tmp.(string)
}

// Removes item under key "foo"
map.Remove("foo")

2 Comments

Things like this are why I can't take seriously the notion that go "doesn't need generics."
The notion isn't that Go "doesn't need generics" but "there is currently no clean way to implement generics, we need to think about this some more". C++ for example generates code for all possible combination of types that are being used, which increases compilation times and executable sizes by an unreasonable amount.
3

if you only have one writer, then you can probably get away with using an atomic Value. The following is adapted from https://golang.org/pkg/sync/atomic/#example_Value_readMostly (the original uses locks to protect writing, so supports multiple writers)

type Map map[string]string
    var m Value
    m.Store(make(Map))

read := func(key string) (val string) { // read from multiple go routines
            m1 := m.Load().(Map)
            return m1[key]
    }

insert := func(key, val string) {  // update from one go routine
            m1 := m.Load().(Map) // load current value of the data structure
            m2 := make(Map)      // create a new map
            for k, v := range m1 {
                    m2[k] = v // copy all data from the current object to the new one
            }
            m2[key] = val // do the update that we need (can delete/add/change)
            m.Store(m2)   // atomically replace the current object with the new one
            // At this point all new readers start working with the new version.
            // The old version will be garbage collected once the existing readers
            // (if any) are done with it.
    }

Comments

1

Why no made use of Go concurrency model instead, there is a simple example...

type DataManager struct {
    /** This contain connection to know dataStore **/
    m_dataStores map[string]DataStore

    /** That channel is use to access the dataStores map **/
    m_dataStoreChan chan map[string]interface{}
}

func newDataManager() *DataManager {
    dataManager := new(DataManager)
    dataManager.m_dataStores = make(map[string]DataStore)
    dataManager.m_dataStoreChan = make(chan map[string]interface{}, 0)
    // Concurrency...
    go func() {
        for {
            select {
            case op := <-dataManager.m_dataStoreChan:
                if op["op"] == "getDataStore" {
                    storeId := op["storeId"].(string)
                    op["store"].(chan DataStore) <- dataManager.m_dataStores[storeId]
                } else if op["op"] == "getDataStores" {
                    stores := make([]DataStore, 0)
                    for _, store := range dataManager.m_dataStores {
                        stores = append(stores, store)
                    }
                    op["stores"].(chan []DataStore) <- stores
                } else if op["op"] == "setDataStore" {
                    store := op["store"].(DataStore)
                    dataManager.m_dataStores[store.GetId()] = store
                } else if op["op"] == "removeDataStore" {
                    storeId := op["storeId"].(string)
                    delete(dataManager.m_dataStores, storeId)
                }
            }
        }
    }()

    return dataManager
}

/**
 * Access Map functions...
 */
func (this *DataManager) getDataStore(id string) DataStore {
    arguments := make(map[string]interface{})
    arguments["op"] = "getDataStore"
    arguments["storeId"] = id
    result := make(chan DataStore)
    arguments["store"] = result
    this.m_dataStoreChan <- arguments
    return <-result
}

func (this *DataManager) getDataStores() []DataStore {
    arguments := make(map[string]interface{})
    arguments["op"] = "getDataStores"
    result := make(chan []DataStore)
    arguments["stores"] = result
    this.m_dataStoreChan <- arguments
    return <-result
}

func (this *DataManager) setDataStore(store DataStore) {
    arguments := make(map[string]interface{})
    arguments["op"] = "setDataStore"
    arguments["store"] = store
    this.m_dataStoreChan <- arguments
}

func (this *DataManager) removeDataStore(id string) {
    arguments := make(map[string]interface{})
    arguments["storeId"] = id
    arguments["op"] = "removeDataStore"
    this.m_dataStoreChan <- arguments
}

1 Comment

This is an original idea, but also illustrates why channel-based approaches make things messy easily.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.