performance - Slow upload with .csv files and PHP -

-

i've created function upload csv files using php , insert contents mongo database.

the upload taking time completed , don't have idea how build better code this.

bellow function in php:

public function importemail($file,$list_name,$header){      $data = $this->readcsv($file, $has_header);             global $email;     $falhas = 0;      array_shift($data);      if($data === false) {         return false;     } else {                           foreach($data $key => $value) {             $email->email = $value[$header["email"]];             $email->first_name = $value[$header["fname"]];             $email->last_name = $value[$header["lname"]];             $email->gender = strtoupper($value[$header["gender"]]);             $email->status = $value[$header["status"]] !="" ? $value[$header["status"]] : "1";             $email->list_name = $list_name;             $email->opt_out = $v = (strtoupper($value[$header["opt"]]) != "" ? strtoupper($value[$header["opt"]]) : "n");                            if(!$email->update(trim($value[$header["email"]]))) $falhas++;                       }           }                return $falhas === 0 ? true : false; }

edited update() method responsible insert records mongodb

public function update($id) {     global $mongo, $gclient;      $update = array(         '$set' => array(             'userdata.name' => $this->first_name,             'userdata.lastname' => $this->last_name,             'userdata.gender' => $this->gender,             'userdata.optout' => $this->opt_out == "y" || $this->opt_out == "n" ? $this->opt_out : "y",             'userdata.lastupdate' => (int)(microtime(true) * 1000),             'userdata.status' => $this->status == 0 || $this->status == 1 ? +$this->status : 0         ),         '$setoninsert' => array(             'userdata.registerdate' => (int)(microtime(true) * 1000)         )     );      if(is_string($this->list_name) && strlen($this->list_name) > 0)         $update['$push'] = array(             'userdata.meta' => array(                 'type' => 'listname',                 'name' => $this->list_name             )         );      $result = $mongo->{$gclient->slug}->usershistory->update(array(         'useremail' => $id     ), $update, array(         'upsert' => true     ));      if(!$this->email) $this->email = $id;      $mongo->{$gclient->slug}->usershistory->update(array(         'useremail' => $id     ), array(         '$set' => array(             'useremail' => $this->email         )     ));      return $result['n'] > 0; }

those .csv files bigger 50k (lines).

how can build faster method?


Comments

Post a Comment

Popular posts from this blog

PHP DOM loadHTML() method unusual warning -

-
i had following code works fine on localhost: $document = new domdocument(); $document->loadhtml($p_result); $form = $document->getelementsbytagname('form')->item(0); // code continues using $form variable after same code updated on outside server, loadhtml() failed , issued warning. warning: domdocument::loadhtml() expects parameter 1 valid path, string given in path/name/to/script.php returned null instead of object code pretty gets fatal error. note contents of $p_result absolutely same on outside server , on localhost. but why displays kind of warning , why not work? doesn't loadhtml() expects argument 1 string in first place? why method expects parameter 1 valid path ? just make clear i'm not calling loadhtmlfile() , i'm calling loadhtml() . thanks. you're affected one of php bugs . issue present in php 5.6.8 , 5.6.9. have affected php version on server, , bug-free version on localhost. the bug forbids null
Read more

python - How to create jsonb index using GIN on SQLAlchemy? -

-
here's current code creating index jsonb. index("mytable_data_idx_id_key", mytable.data['id'].astext, postgresql_using='gin') but got error. sqlalchemy.exc.programmingerror: (psycopg2.programmingerror) data type text has no default operator class access method "gin" hint: must specify operator class index or define default operator class data type. [sql: "create index event_data_idx_id_key on event using gin ((data ->> 'id'))"] is there way create index on sqlalchemy? the postgresql specific sqlalchemy docs @ http://docs.sqlalchemy.org/en/latest/dialects/postgresql.html#operator-classes mention postgresql_ops dictionary provide "operator class" used postgresql, , provide example illustrating use: index('my_index', my_table.c.id, my_table.c.data, postgresql_ops={ 'data': 'text_pattern_ops',
Read more

c# - TransactionScope not rolling back although no complete() is called -

-
i'm using transactionscope rollback transaction fail bool errorreported = false; action<importerrorlog> newerrorcallback = e => { errorreported = true; errorcallback(e); }; using (var transaction = new transactionscope()) { foreach (importtaskdefinition task in taskdefinition) { loader.load(streamfile, newerrorcallback, task.destinationtable, processingtaskid); } if (!errorreported) transaction.complete(); } i'm sure there no transactionscope started ahead or after code. i'm using entity framework insert in db. regardless state of errorreported transaction never rolled in case of error. what missing ? transactionscope sets transaction.current . that's does. wants transacted must @ property. i believe ef each time connection opened reason. probably, connection open when scope installed. open connection inside of scope or enlist manually. ef has nasty "design decision": default opens
Read more