On 6/8/06, Kevin Closson <kevinc@(protected)> wrote: > > Storage: Dell Clariion CX700 SAN > > ...powerpath ?
Dunno yet, but I think so.
Errors in file > /u01/app/oracle//admin/pr09/bdump/pr09_ckpt_29193.trc: > ORA-01171 (See ORA-01171.ora-code.com): datafile 12 going offline due to error advancing > checkpoint > ORA-01110 (See ORA-01110.ora-code.com): data file 12: > '/u01/oradata/pr09/agile_indx101pr09.ora' > ^^^^^^^^^^^^^ > ... Is this ext3 ?
Yes, this is ext3.
ORA-01115 (See ORA-01115.ora-code.com): IO error reading block from file 12 (block # 1) > ORA-27072 (See ORA-27072.ora-code.com): skgfdisp: I/O error > Linux-x86_64 Error: 5: Input/output error > > ... are there corresponding /var/log/messages entries for these EIO > errors tht would impugn the scsi layer or other goodies downwind? Trying > to establish if this is an EIO pushed up from a software layer (perhaps > powerpath) ? Being intermittant there whould be /var/log/messages > goodies to clue us in
No IO error messages that I can find yet, but did find something else:
Jun 7 23:15:07 ordb02 kernel: ctxhx[3472]: segfault at 00000000000007b5 rip 00000000f7d240f6 rsp 00000000ffffa528 error 4 Jun 7 23:17:53 ordb02 kernel: ctxhx[3972]: segfault at 00000000000007b5 rip 00000000f7d240f6 rsp 00000000ffffa528 error 4 Jun 7 23:31:39 ordb02 kernel: ctxhx[6551]: segfault at 00000000f790c02c rip 00000000f7ea39c3 rsp 00000000ffffa370 error 6 Jun 8 09:55:05 ordb02 kernel: ctxhx[18246]: segfault at 00000000000007b5 rip 00000000f7d240f6 rsp 00000000ffffb1b8 error 4
There are 2 other databases running the same app, same versions of Oracle, so it is a little difficult to be sure which is causing the problem.
Since most of these are the past 2 days, it could be that this is just a symptom. The IO errors may have crashed ctxhx (I'm assuming it has something to do with building Text indexes)
-- Jared Still Certifiable Oracle DBA and Part Time Perl Evangelist
On 6/8/06, <b class="gmail_sendername">Kevin Closson</b> <<a href="mailto :kevinc@(protected)">kevinc@(protected)</a>> wrote:<div><span class= "gmail_quote"></span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> Storage: Dell Clariion CX700 SAN<br><br>...powerpath ?</blockquote><div><br>Dunno yet, but I think so.<br><br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> Errors in file<br>/u01/app /oracle//admin/pr09/bdump/pr09_ckpt_29193.trc:<br> ORA-01171 (See ORA-01171.ora-code.com): datafile 12 going offline due to error advancing<br >checkpoint<br> ORA-01110 (See ORA-01110.ora-code.com): data file 12:<br>'/u01/oradata/pr09/agile_indx101pr09.ora' <br> ^^^^^^^^^^^^^<br>...   ; Is this ext3 ?</blockquote><div><br><br>Yes, this is ext3.<br> </div> <br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> ORA-01115 (See ORA-01115.ora-code.com): IO error reading block from file 12 (block # 1)<br> ORA-27072 (See ORA-27072.ora-code.com): skgfdisp: I/O error<br> Linux-x86_64 Error: 5: Input/output error<br><br>... are there corresponding /var/log/messages entries for these EIO <br>errors tht would impugn the scsi layer or other goodies downwind? Trying<br >to establish if this is an EIO pushed up from a software layer (perhaps<br >powerpath) ? Being intermittant there whould be /var/log/messages <br>goodies to clue us in</blockquote><div><br><br>No IO error messages that I can find yet, but did find something else:<br><br>Jun 7 23:15:07 ordb02 kernel: ctxhx[3472]: segfault at 00000000000007b5 rip 00000000f7d240f6 rsp 00000000ffffa528 error 4 <br>Jun 7 23:17:53 ordb02 kernel: ctxhx[3972]: segfault at 00000000000007b5 rip 00000000f7d240f6 rsp 00000000ffffa528 error 4<br>Jun 7 23:31:39 ordb02 kernel: ctxhx[6551]: segfault at 00000000f790c02c rip 00000000f7ea39c3 rsp 00000000ffffa370 error 6 <br>Jun 8 09:55:05 ordb02 kernel: ctxhx[18246]: segfault at 00000000000007b5 rip 00000000f7d240f6 rsp 00000000ffffb1b8 error 4<br><br>There are 2 other databases running the same app, same versions of Oracle, so it<br >is a little difficult to be sure which is causing the problem. <br><br>Since most of these are the past 2 days, it could be that this is just a symptom. The IO errors<br>may have crashed ctxhx (I'm assuming it has something to do with building Text indexes)<br><br></div></div>-- <br> Jared Still<br>Certifiable Oracle DBA and Part Time Perl Evangelist<br>
