php - Recursive function until done value is true and merge each response into a big one -

-

i have function:

public function syncterritoriessoqlquery($veevatoken, $instanceurl, $tokenurl) {     $soqlquery2 = "select id,name,lastmodifieddate territory";     $soqlurl2 = $instanceurl.'/services/data/v28.0/query/?q='.urlencode($soqlquery2);      $curl = curl_init($soqlurl2);      curl_setopt($curl, curlopt_header, false);     curl_setopt($curl, curlopt_returntransfer, true);     curl_setopt($curl, curlopt_httpheader, array("authorization: oauth $veevatoken"));      $jsonresponse = curl_exec($curl);      $status = curl_getinfo($curl, curlinfo_http_code);     if ($status !== 200) {         $respobj['error'] = "error: call token url $tokenurl failed status $status, response $jsonresponse, curl_error ".curl_error(                 $curl             ).", curl_errno ".curl_errno($curl);          return $respobj;     }      curl_close($curl);      $soqlobj2 = json_decode($jsonresponse, true);      return $soqlobj2; }

when call got response:

{   "totalsize": 6911,   "done": false,   "nextrecordsurl": "/services/data/v28.0/query/01g8000002ei8dmaas-2000",   "records": [ ... ] }

that means complete set of records have 6911 items on first request first 2000 return since it's paginated. need call same code again , again , again until "done": true. in every new request need change $soqlurl2 append value return on each call. example:

1st time: $soqlurl2 = $instanceurl.'/services/data/v28.0/query/?q='.urlencode($soqlquery2);  2nd time: $soqlurl2 = $instanceurl."/services/data/v28.0/query/01g8000002ei8dmaas-2000";      3rd time: $soqlurl2 = $instanceurl."/services/data/v28.0/query/01g8000002ei8dmaas-4000";  ...

that until done gets true on response. should merge whole json result each iteration big one, lets said: result 2nd time should merged result first call meaning $soqlobj2 result 3d time should merge previous merge in pseudo-code:

merge($soqlobj2, 1st, 2nd, 3rd, ...)

i have been working on function:

public function performpaginatesoqlquery($veevatoken, $instanceurl, $tokenurl, $nextrecordsurl, &$datatosave = array()) {     if ($nextrecordsurl !== null && $nextrecordsurl !== "") {         $keeprequesting = true;          while($keeprequesting){             $nextrecordsurlsoqlquery = $instanceurl.$nextrecordsurl;              $curl = curl_init($nextrecordsurlsoqlquery);              curl_setopt($curl, curlopt_header, false);             curl_setopt($curl, curlopt_returntransfer, true);             curl_setopt($curl, curlopt_httpheader, array("authorization: oauth $veevatoken"));              $jsonresponse = curl_exec($curl);              $status = curl_getinfo($curl, curlinfo_http_code);             if ($status != 200) {                 $respobj['error'] = "error: call token url $tokenurl failed status $status, response $jsonresponse, curl_error ".curl_error(                         $curl                     ).", curl_errno ".curl_errno($curl);                  return $respobj;             }              curl_close($curl);              $nextrecordsurlobj = json_decode($jsonresponse, true);             $datatosave[] = $nextrecordsurlobj;              echo "iteration 2", "\n";             echo "url: ", $nextrecordsurl, "\n";             echo "done: ";             var_dump($nextrecordsurlobj['done']);             echo "\n";              if($nextrecordsurlobj['done'] === true) {                 $keeprequesting = false;             }         }          return array('url' => $nextrecordsurlobj, 'data' => $datatosave);     } }

and call function above syncterritoriessoqlquery() follow:

$entiresoqlobj2 = $this->performpaginatesoqlquery(         $veevatoken,         $instanceurl,         $tokenurl,         $soqlobj2['nextrecordsurl'] );

but not working since getting result:

iteration 0 url: /services/data/v28.0/query/01g8000002eyb8laas-2000 done: bool(false)  iteration 2 url: /services/data/v28.0/query/01g8000002eyb8laas-2000 done: bool(false)  iteration 2 url: /services/data/v28.0/query/01g8000002eyb8laas-2000 done: bool(false)  iteration 2 url: /services/data/v28.0/query/01g8000002eyb8laas-2000 done: bool(false)  iteration 2 url: /services/data/v28.0/query/01g8000002eyb8laas-2000 done: bool(false)

right output should like:

// 1st url: /services/data/v28.0/query/01g8000002eyb8laas-2000 done: bool(false)  // 2st url: /services/data/v28.0/query/01g8000002eyb8laas-4000 done: bool(false)  // 3rd url: /services/data/v28.0/query/01g8000002eyb8laas-6000 done: bool(true)

because totalsize 6911, can me fix code? mistake?

note: ask this earlier today not clear me @ i've open new question more info.

you should have $soqlobj2['nextrecordsurl'] inside recursive function , should updated $nextrecordsurl every time. checking first page , hence getting url second pages.

also, don't understand why have code in performpaginatesoqlquery() , want iterate on syncterritoriessoqlquery() until done, right?

update:

updating syncterritoriessoqlquery() loop on pages. added comments everywhere made changes. hope helps.

<?php public function syncterritoriessoqlquery($veevatoken, $instanceurl, $tokenurl) {     $soqlquery2 = "select id,name,lastmodifieddate territory";      // removed $instanceurl, we'll add inside while loop     $instancepath = '/services/data/v28.0/query/?q='.urlencode($soqlquery2);      $final_output = array(); // output appended array     $done = false; // set variable true, once `done` true      while( !$done ) {         $soqlurl2 = $instanceurl . $instancepath;          $curl = curl_init($soqlurl2);          curl_setopt($curl, curlopt_header, false);         curl_setopt($curl, curlopt_returntransfer, true);         curl_setopt($curl, curlopt_httpheader, array("authorization: oauth $veevatoken"));          $jsonresponse = curl_exec($curl);          $status = curl_getinfo($curl, curlinfo_http_code);         if ($status !== 200) {             $respobj['error'] = "error: call token url $tokenurl failed status $status, response $jsonresponse, curl_error ".curl_error(                     $curl                 ).", curl_errno ".curl_errno($curl);              return $respobj;         }          curl_close($curl);          $soqlobj2 = json_decode($jsonresponse, true);          // here, change $instancepath nextrecordsurl         $instancepath = $soqlobj2['nextrecordsurl'];          //  set $done variable         $done = $soqlobj2['done'];          // appending data obtained $final_output array, can change string if u need.         $final_output[] = $soqlobj2;        }      return $final_output; }

Comments

Post a Comment

Popular posts from this blog

python - How to create jsonb index using GIN on SQLAlchemy? -

-
here's current code creating index jsonb. index("mytable_data_idx_id_key", mytable.data['id'].astext, postgresql_using='gin') but got error. sqlalchemy.exc.programmingerror: (psycopg2.programmingerror) data type text has no default operator class access method "gin" hint: must specify operator class index or define default operator class data type. [sql: "create index event_data_idx_id_key on event using gin ((data ->> 'id'))"] is there way create index on sqlalchemy? the postgresql specific sqlalchemy docs @ http://docs.sqlalchemy.org/en/latest/dialects/postgresql.html#operator-classes mention postgresql_ops dictionary provide "operator class" used postgresql, , provide example illustrating use: index('my_index', my_table.c.id, my_table.c.data, postgresql_ops={ 'data': 'text_pattern_ops', ...
Read more

PHP DOM loadHTML() method unusual warning -

-
i had following code works fine on localhost: $document = new domdocument(); $document->loadhtml($p_result); $form = $document->getelementsbytagname('form')->item(0); // code continues using $form variable after same code updated on outside server, loadhtml() failed , issued warning. warning: domdocument::loadhtml() expects parameter 1 valid path, string given in path/name/to/script.php returned null instead of object code pretty gets fatal error. note contents of $p_result absolutely same on outside server , on localhost. but why displays kind of warning , why not work? doesn't loadhtml() expects argument 1 string in first place? why method expects parameter 1 valid path ? just make clear i'm not calling loadhtmlfile() , i'm calling loadhtml() . thanks. you're affected one of php bugs . issue present in php 5.6.8 , 5.6.9. have affected php version on server, , bug-free version on localhost. the bug forbids null ...
Read more

c# - TransactionScope not rolling back although no complete() is called -

-
i'm using transactionscope rollback transaction fail bool errorreported = false; action<importerrorlog> newerrorcallback = e => { errorreported = true; errorcallback(e); }; using (var transaction = new transactionscope()) { foreach (importtaskdefinition task in taskdefinition) { loader.load(streamfile, newerrorcallback, task.destinationtable, processingtaskid); } if (!errorreported) transaction.complete(); } i'm sure there no transactionscope started ahead or after code. i'm using entity framework insert in db. regardless state of errorreported transaction never rolled in case of error. what missing ? transactionscope sets transaction.current . that's does. wants transacted must @ property. i believe ef each time connection opened reason. probably, connection open when scope installed. open connection inside of scope or enlist manually. ef has nasty "design decision": default opens ...
Read more