Skip to content

Move to memory db - #5

Open
sternj wants to merge 8 commits into
masterfrom
moveToMemoryDb
Open

sternj wants to merge 8 commits into
masterfrom
moveToMemoryDb

Conversation

@sternj

@sternj sternj commented Dec 27, 2019

Copy link
Copy Markdown
Member

No description provided.

@hojulian hojulian left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like where this is going, looks good for most of it. Missing the crucial StartTask part but that's it. Good work!

Comment on lines +7 to +15
"github.com/Capstone-auto-grader/grader-api-v2/internal/grader-task"
"github.com/Capstone-auto-grader/grader-api-v2/internal/graderd"
"github.com/Capstone-auto-grader/grader-api-v2/internal/sync-map"
"github.com/docker/docker/api/types"
"github.com/docker/docker/client"
"io/ioutil"
"log"

"github.com/pkg/errors"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ same for other files as well

Comment thread cmd/graderd/main.go Outdated
Comment on lines +19 to +20
"github.com/Capstone-auto-grader/grader-api-v2/internal/docker-client"
sync_map "github.com/Capstone-auto-grader/grader-api-v2/internal/sync-map"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread cmd/graderd/main.go Outdated
}
grpcServer := grpc.NewServer(grpc.Creds(serverCert))
graderService := graderd.NewGraderService(graderd.NewDockerClient(*dockerAddr, *dockerVersion), graderd.NewPGDatabase(*databaseAddr), *webAddr)
graderService := graderd.NewGraderService(docker_client.NewDockerClient(*dockerAddr, *dockerVersion, sync_map.NewSyncMap(), 2), *webAddr)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure about creating and passing the sync_map in the constructor of docker_client, any reason why we don't just create it inside the constructor?

Comment on lines +12 to +14
_ = tasks.UpdateStatus(taskId, grader_task.StatusStarted, true)
_ = client.StartTask(context.Background(), taskId)
_,_ = client.TaskOutput(context.Background(), taskId)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can make another channel here for propagating errors out.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking a channel for all responses and errors, so another worker can deal with them

Comment thread internal/docker-client/docker.go Outdated
for _, t := range tasks {
if t.ContainerID == c.ID {
status := graderd.ParseContainerState(c.State)
if status == grader_task.StatusStarted {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we show the tasks that stopped unexpectedly as well, just a thought.

Comment thread internal/sync-map/map.go
"sync"
"github.com/Capstone-auto-grader/grader-api-v2/internal/grader-task"
)
// A synchronized map to store Tasks

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment should start with "SyncMap is ..."/"SyncMap ... ": https://golang.org/doc/effective_go.html#commentary

Comment thread internal/docker-client/docker.go Outdated
type DockerClient struct {
cli *client.Client
mp *sync_map.SyncMap
queue chan<- string

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make this two-way, otherwise you won't be able to pass stuff in.

//}

func (d *DockerClient) StartTask(ctx context.Context, taskID string) error {
return nil

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the important part is missing!!!

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm hammering that one out after I write the rest of it!

Comment thread internal/sync-map/map.go Outdated
defer m.mu.Unlock()
t := m.mp[taskID]
if checkStatus && t.Status == grader_task.StatusStarted {
return fmt.Errorf("task already started")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use errors.New(...) instead

Comment thread internal/sync-map/map.go
return nil
}

func (m *SyncMap) Enumerate() map[string]grader_task.Task {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im not very sure about this approach. Can you explain more on why we need to copy the whole map everytime? When the use-case is just readonly?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm worried about a concurrent update to some of the elements of the task (status or containerID) in the middle of reading. Thinking about it more, I don't think it would be the worst thing in the world to have an intermediate state read, what do you think @xumr0x ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants